SQL 2016 - New Features

1. Always Encrypted
2. JSON Support
3. Dynamic Data Masking
4. PolyBase
5. Row Level Security
6. Stretch Database
7. Multiple TempDB
8. Query Store
9. Temporal Table
10. R Introduction

Introduction

SQL Server 2016 was (finally) released on June 1st, 2016 with an initial build number of
13.0.1601.5. Microsoft build SQL 2016 keeps a lot of things in mind like Cloud first,
Security enhancement, JSON support, Temporal database support, Row level security,
Windows server 2016 connectivity, Non-relational database connectivity (e.g. Hadoop),
rich visual effects, etc.

In this article, we will take a walk-through all fresh SQL 2016 features and cover them
one by one.
Always Encrypted

As the word suggests, 'Always Encrypted' feature of SQL 2016 'Always' keeps your
sensitive data 'Encrypted' either at rest (local environment) or at remote (Cloud/Azure).
It will help to protect data from people who may play around it like DBAs, Cloud
operators, high-privileged but unauthorized users.

How It Works

You can set Always Encrypted to individual column (where your sensitive data resides).
While configuring columns, you need to specify encryption algorithm and cryptographic
keys for data protection. There are basically two keys you need to define:

1. Encryption Key for column data encryption (It will be used to encrypt data for specific
column)
2. Master Key: for Encryption of column encryption keys

So basically, it's a double encryption protection, only program can access it, client
application will automatically encrypt and decrypt data before fetching data from
database.

JSON Support
SQL 2016 gives direct support to JSON (Java Script Object Notation), SQL has the facility
to read JSON format data, load them in table, support index properties in JSON
columns.

JSON data will be stored in NVARCHAR type. Due to NVARCHAR type, an application has the
following benefits:

 Already stored JSON data (as text) can be easily migrated on new feature.
 As NVARCHAR is supported by all SQL components so is the JSON too.

You can easily fetch data FOR JSON from SQL with the below syntax:

SQL

SELECT column, expression, column as alias

FROM table1, table2, table3
FOR JSON [AUTO | PATH]

It is a SELECT command so when we fire the above query, SQL will format each row/cell
value and return as JSON object.

SQL has also provided in-built functions for JSON.

Dynamic Data Masking

This is again one of the security features of SQL 2016. As the name suggests, it just
wraps MASK on your data, in short, it hides your confidential data that you don't want
to display. It just avoids disclosure of your sensitive data.
After masking, SQL User with limited rights will not view original text, he can view only
Masked Text, SQL has pre-defined masking functions you just need to apply it on
different columns, see below:

Sr No Functions Applied on Plain text (Input) Masking text(output)

1 Default String, Number ABCD xxxx

2 Email Email text [email protected] [email protected]

3 Random Numbers 1234 684

4 Custom String Text RABBIT RXXXX

To apply this on specific columns, you just need to ALTER column with 'MASKED WITH'
function name, see below syntax:

SQL

//here I used function as Default(), you can change it to any of the above types

ALTER TABLE tablename ALTER COLUMN columnname MASKED WITH (FUNCTION=‚default()‘)

PolyBase

It's a multi-connection functionality, in which we can connect to all relational and non-
relational data from single point, it helps you to connect Hadoop database and Azure
Blob storage. Basically, PolyBase creates a bridge between a data that is outside SQL
scope, while querying on Hadoop or Azure storage no additional knowledge or
installation is needed. Simply, you can Import and Export data To and From Hadoop or
Azure storage. Additionally, you can integrate it with Microsoft’s business intelligence.

Row Level Security

This is again one of the security features of SQL 2016. It allows you to secure your data
row wise, in short you can define a row, that will be viewed by a particular SQL user only.
So depending upon the SQL user access permission, we can restrict row level data, e.g.,
we can ensure if employees can view only their department data though department
table is the same.

To implement Row level security, you need to define Security policy with a predicate and
function.

Security policy: We need to create a policy for security, here is simple syntax:

SQL

CREATE SECURITY POLICY fn_security ADD [FILTER | BLOCK] PREDICATE FunctionName ON

TableName

In the above syntax, FILTER and BLOCK are the predicates that will either FILTER rows
and display only those that are available for read or BLOCK rows for write operation.

Function: Function is a simple user defined function, but here are some restrictions for
user defined function that are used in Row Level Security syntax:

 Database modification operations are not allowed

 OUTPUT INTO clause is not allowed in function
 Multiple result set should not be returned from function
Stretch Database

As the name suggests, it gives flexibility to the user. In short, we can store portion of
database to remote (Here, we can say cloud/Azure). The portion of data can be called as
COLD DATA. (It is useful for those where transactional data needs to be keep for long
time as industry requirement.) So we can say it's a cost-effective solution for COLD data
storage, your data is available anytime for query and manage. You can access your data
without changing queries either it is present on local or at stretch database.

To configure it, you need an Azure account and database instance that you need to
stretch. The following snap will clear your idea.

Multiple TempDB
It is always a good practice to have a Multiple Temp data files, if you are working on a
big crucial data, up till now (SQL 2014), you need to manually add temp db files to your
database but SQL 2016 provides you temp DB configuration settings, in which you can
configure Number of TempDB files at the time of SQL installation. Default number of
files are 8 with default size of 64 MB will be given.

So you no longer need to configure/create it manually.

Query Store

Up till now, to check Query plan and execution statistics, we need dynamic management
views in SQL but neither will it give you Query plan that executed by past/old queries
nor will it store them anywhere so that you can review, but SQL 2016 provides you
'Query Store' that takes you through query plan, statistics and Query execution for
current and past queries.

To enable it, just right click on database (obviously, you need SQL 2016 SSMS), go to
properties. You will see 'Query store' at the left corner, select it and click on Enable
'true' or you can do it using Query as follows:

SQL

ALTER DATABASE [Database1] SET QUERY_STORE = ON

Temporal Table

Do you want to store history of your SQL table? So you want to review your old records
after table updation? Then you can go with this features. SQL 2016 provides record
version facility in which it keeps a history of changed record and maintains it for timely
analysis. This feature can be useful for Audit, checking data trend, accidental
update/delete data and many more.

How It Works

Basically, the system keeps pair of a table for history and adds two additional columns in
it named 'SysStartTime' and 'SysEndTime' for start time and end time for row
respectively. Live table contains current record of row, whereas history table contains
previous record of row. We can fetch data from History table, with the following query:

SQL

SELECT * FROM table1 FOR SYSTEM_TIME

BETWEEN date1 AND date2
WHERE condition;

R Introduction

Have you stored statistical data in SQL? Want to use R to analyze it? You export data each
time from SQL to R? Then your headache will now be covered in SQL 2016, because it is
now with R. You can run R script on SQL. You need to install this feature at the time of
SQL setup.
New Features of SQL Server 2019
Intelligent Query Processing Enhancements
Accelerated Database Recovery (ADR)
AlwaysEncrypted with secure enclaves
Memory-optimized Tempdb metadata
Query Store custom capture policies

Verbose truncation warnings

Resumable index build

Data virtualization with Polybase

Last actual execution plan DMF

UTF-8 Support
Min Memory and Max Memroy, MAXDOP setting during the command prompt
instlaltion.

1. Intelligent Query
Processing Enhancements
What they are: This is a set of enhancements that affect the behavior of the
Query Optimizer; the component inside SQL Server that generates the execution
plans for queries. This includes dynamic memory grants for rowstore tables, table
variable deferred compilation, batch mode on rowstore and more.
Why this matters: These are all behind-the-scenes improvements on the Query
Optimizer that will improve the quality of the plans for all applicable queries. This
means better performance overall after doing the upgrade.
Cost of adoption: Test your problem queries on a development instance to
verify the improvements. That’s pretty much it. It’s one of those great
improvements that works with no big changes required from the customer (I
really like those as you will see from this list).

2. Accelerated Database
Recovery (ADR)
What this is: This is a completely new way for SQL Server to perform database
recovery in the event of a transaction rollback, an instance restart or an
availability group failover. Instead of the sometimes-unpredictable and less-than-
desired time spent waiting for the database recovery to run, the SQL team has
redeveloped how recovery works and has dramatically decreased how long this
process takes.
Why this matters: Anyone who has had to wait for a production SQL Server
instance to rollback a long transaction or who has had an unfortunate crash
during a large data operation knows how painful it is to just wait for recovery to
be finished so you can get back in business. ADR will provide benefits for SLA
and RTO all around.
Cost of adoption: None. Activate it and enjoy (one of those again!).

3. AlwaysEncrypted with
secure enclaves
What this is: This is the next version of AlwaysEncrypted; the encryption
technology introduced in SQL Server 2016 that allows transparent column
encryption without giving administrators access to the decryption keys. One
drawback of the first implementation is due to SQL Server not being able to
decrypt the data, the queries on the SQL side couldn’t do any computations or
manipulation of the actual column values. Using the new secure enclaves
technology, SQL Server can now securely encrypt a portion of memory to
perform computations on these encrypted columns without ever exposing the
unencrypted values to the rest of the processes (or administrators).
Why this matters: Security matters and performance matters, as well. Database
servers are best equipped for processing large amounts of data, so being able to
have AlwaysEncrypted and also do complex manipulations is the best of both
worlds.
Cost of adoption: If you’re already using AlwaysEncrypted then no big changes
are necessary other than reconfiguring and re-encrypting the columns of interest.
If you aren’t using AlwaysEncrypted, now is a good time to investigate, test this
feature and see if it’s a good fit for your security requirements.

4. Memory-optimized
Tempdb metadata
What this is: The SQL team has made optimizations to the tempdb code so
some of the metadata that can be a bottleneck on tempdb heavy systems can
rely completely on memory and be optimized for RAM access.
Why this matters: Large-volume, large-scale environments that use a lot of
tempdb run into this type of bottleneck. Usually, alleviating the use of tempdb
requires some sort of refactoring. With this feature in place, it is possible to
enable the metadata to sit in memory and be optimally accessed.
Cost of adoption: Activate the feature and verify there is an improvement, not
much more than that (another one!).
5. Query Store custom
capture policies
What this is: Query Store is a great performance tuning and trending tool that
allows for storing, measuring and fixing plan regressions inside a SQL Server
database. One downside of using it, however, is it can sometimes store too much
information, even for queries the DBA might not be interested in or for queries
that were part of a system utility or monitoring tool. This new capability of
custom policies means you can fine-tune exactly which queries you want Query
Store to track based on their execution statistics such as how often they run, the
CPU they consume and more.
Why this matters: Query Store is a great feature but it is not so useful if it
consumes a lot of resources and if it’s too bloated to be effective for the DBA.
This feature will allow fine-tuning so it’s always efficient, lean and easy to use for
fixing plan issues.
Cost of adoption: You’ll need to sit down and see what type of execution
conditions you want to use as a filter for your Query Store. Implementing it is just
a matter of using new syntax after that.

6. Verbose truncation
warnings
What this is: Every single T-SQL developer knows the pain and grind of getting a
truncation error. Some value somewhere doesn’t fit under a new data type, but
you don’t get any details at all. Then it’s a matter of trial and error until you finally
figure out which value is the offending one. Not the best experience for what
should be a straightforward issue to solve!
Why this matters: Because mental sanity matters. These new messages give you
all the details of the data truncation issue so you can just fix it and get on with
your day!
Cost of adoption: None. It’s the new default (loving all of these)!
7. Resumable index build
What this is: SQL Server now has the capability to stop an index rebuild
operation in progress, keep the work that has been done so far, and resume at
some other point in time.
Why this matters: For some folks, index rebuilds are still necessary and they
consume so many resources that even with the ONLINE option, they still have to
deal with the reality of maintenance windows. The problem, however, is what
happens if you run out of time during your maintenance window? Previously, you
would need to cancel your rebuild, wait for a potentially long recovery, then start
again from scratch. This new feature gets rid of these problems!
Cost of adoption: Change your index scripts to use the new RESUMABLE option.
Pretty easy (another one!).

8. Data virtualization with

Polybase
What this is: Polybase is SQL Server’s module that allows fast and parallel T-SQL
queries that can go out into external storage (usually HDFS on-prem) and return
results seamlessly as a T-SQL result set. With SQL 2019, Polybase is getting
expanded to support Oracle, Teradata, MongoDb and more.
Why this matters: Data integration is always a challenge, and with ever-growing
data sets, performance can become an issue. Trying to query and move large
amounts of data through a linked server has always been painfully slow as it is
not really optimized for this job. Polybase allows SQL Server to become the data
hub of an organization by leveraging T-SQL skills and keeping performance as a
top priority at the same time.
Cost of adoption: This one definitely requires some work as you’d want multiple
Polybase nodes, and would have to set up connectivity to your other database
platforms, then test the performance of those queries.
9. Last actual execution plan
DMF
What this is: This is a new Dynamic Management Function called
sys.dm_exec_query_plan_stats that will track the last ACTUAL execution plan for a
query if you enable the lightweight query profiling feature (which you probably
should do).
Why this matters: Previously, grabbing an actual query plan required either a
Profiler trace, an XEvents trace, or a call to an ephemeral DMF that would lose its
contents when the query was done executing. Not the easiest or most
convenient mechanisms to do what is pretty much a critical step in any sort of
production performance problem scenario.
Cost of adoption: Enable the setting, use the DMF (another one to just use!).
10. Multiple internal
performance improvements
What this is: The SQL team has made multiple internal performance
improvements for this release. There’s a new index optimization for indexes that
have a sequential key, temp table recompilation improvements, improved
indirect checkpoint scalability and more.
Why this matters: These are all performance improvements that come “out-of-
the-box,” optimize common SQL Server processes and require no effort from the
client to benefit from them.
Cost of adoption: See line above. SCORE!

New Features of SQL Server 2019

 New Features of SQL Server 2019
o Big Data Clusters
o UTF-8 Support (CTP 2.2)
o Resumable online index create (CTP 2.0)
o Intelligent query processing (CTP 2.0)
o Always On availability groups
o Machine Learning on Linux
o New features for SQL Server on Linux
o Security

1. Big Data Clusters

Big data clusters are new additions to the SQL server 2019 release. This feature allows you to
deploy multiple, scalable clusters of SQL Server, Spark, and HDFS containers running on
Kubernetes, at once. The Big data Cluster, as an infrastructure, allows these clusters to run
parallelly, where you can read, write, and process Big Data from Transact-SQL to Spark. It
enables us to easily combine and analyze the high-value relational data with high-volume big
data.

Features:
 Data Virtualization: SQL Server PolyBase has eased the task of querying the external
data sources for the SQL Server big data clusters, by reducing the effort of moving or
copying the data for making a query. SQL Server 2019 preview has introduced new
connectors to data sources.

 Data Lake: The big data cluster allows for a scalable HDFS storage pool. This
potentially increases the efficiency of big data storage from external sources.

 Scale-out data mart: Big data cluster provides scale-out compute and storage to
improve the data analysis. The data can be ingested and stored across multiple data
pool nodes as cache, for further analysis.
  Integrated AI and Machine Learning: The big data cluster allows for AI and ML on
the data stored across multiple HDFS storage pools and data pools. SQL server
provides many built-in AI tools like R, Python, Scala or Java.
 Management and Monitoring: The cluster administrator portal is the website that
provides the status and health of the pods in the cluster. It also provides links for
other dashboards for log analytics and monitoring.
 Management and monitoring will be done using the combination of the command
line tools, APIs, administrator portal and dynamic management
views.

Advantages of Big Data Cluster:

 Has built-in snippets for regular management tasks.

 Allows browsing HDFS, to create directories, to preview files and upload files.
 Allows creating, opening and running Jupyter-compatible notebooks.
 The creation of external data sources has been simplified by the Data Virtualization
Wizard.
 Big data cluster with K8 infrastructure increases the speed of setting up the whole
group infrastructure.
 The security concerns arising with the integration of the relational environment with
the Big Data are handled completely by the big data clusters.
 The data virtualization allows for easy data integration without having to perform ETL
(extract, transform, and load).
2. UTF-8 Support

The new SQL Server 2019 supports the very popular UTF-8 data encoding system. The UTF-8
character encoding is employed in data export, import, database-level, and column -level data
collation. It is enabled when creating or changing the object collation type to object collation
with UTF-8. It is supported for char and varchar data types.

The reason why data has to be encoded while storing and retrieving is mainly because of 2
reasons.

 For reducing the memory occupancy or storage space.

 To provide data security for sensitive data.

Note: As of Microsoft SQL Server 2016, UTF-8 is supported by BCP, BULK_INSERT, and
OPENROWSET.

The earlier versions of SQL Server had encoding done in different formats like UCS-2 and they
did not support the UTF-8 format. However, the introduction of Unicode encoding was done
only from SQL Server 7.0.

Advantages of UTF Encoding:

This feature helps in storage saving, by using the right character set. For example, changing
the existing data type of column with Latin strings from NCHAR(10) to CHAR(10) using a UTF-
8 enabled collation, leads to 50% reduction in storage requirements. This saving happens
because NCHAR(10) requires 20 bytes for storage, whereas CHAR(10) requires 10 bytes for
the same Unicode string.

CTP 2.1 allows selecting UTF-8 collation as default during SQL Server 2019 preview setup.

CTP 2.2 allows selecting to use UTF-8 character encoding with SQL Server Replication.

3. Resumable Online Index Create (CTP 2.0)

This is the feature that allows an index to create operation to pause and resume later, right
from the point where the operation failed or paused, instead of starting the process all over
again.
The index is one of the powerful tools for database management. With more operations on
databases like insert, update and delete, the index becomes more fragmented and hence less
efficient. In order to combat this, index rebuild operations are increasingly adopted by the
DBAs.

Resumable Online Index Rebuilding (ROIR) was adopted from SQL Server 2017 as an
important feature to enhance database performance.

However, in the SQL Server 2019 version, a newer version of the feature is incorporated, which
is "Resumable Online Index Create"

Features of the Resumable Online Index Create

 You can resume the index create operation after an index creates failure in case of
overuse of disk space or during database loss.
 Pausing the ongoing index create operation in case of blockages will result in freeing
up the resources temporarily, to resume the blocked tasks.
 The heavy log generation due to the cumbersome index creation operation can be
handled by pausing the index create operation, truncating or taking the backup of
the log and then resuming the same.

In the older versions, when this feature was not introduced, upon the new index creates
operation failure, the whole process had to start from the beginning.

The SQL Server 2019 also allows setting this as a default feature for a specific database.

4. Intelligent Query Processing (CTP 2.0)

The feature of Intelligent Query Processing (IQP) is a method adopted to obtain an optimal
query execution plan with lower compiler time. This feature is expanded to include many
other sub-features in the SQL Server 2019, CTP 2.2.

There are many factors considered while executing IQP, mainly to generate a good enough
execution plan. These factors are Structures to be used, Joins to be made in a query (Hash
Join, Nested Loop, Merge Adaptive, etc.), Outer Input, execution mode (Batch or Row
execution mode), etc.

SQL Server 2017 had the feature of Intelligent Query Processing with the following sub-
features:
  Adaptive joins in Batch Mode to dynamically select a join type during the runtime or
execution time, based on the input rows.
 Interleaved Execution under the Compatibility Level 140, which uses the cardinality
of a multi-statement table according to the values encountered on the first
compilation, rather than a fixed guess.
 Memory Grant Feedback (Batch mode) to handle the memory allocation. If a batch
mode query has operations that demand extra disk space, more memory will be
allocated for it, in the consecutive executions. While, if the query uses less than 50%
of the allocated memory, the memory grant will be reduced from the consecutive
executions.

There are, however, many improvements made in the IQP for the SQL Server 2019 CTP 2.0
preview version. These features are:

 Starting from SQL Server 2019 CTP 2.0, the server provides an approximate Count
Distinct for the big data scenarios. Count Distinct returns the approximate number
of unique non-null values in a group. This feature reduces the memory footprint,
hence increasing performance efficiency.
 Batch mode on Row store is allowed in the 2019 version, under compatibility level
150, which provide the batch mode on for the CPU bound relational DW workloads.
This feature does not require having column store indexes.
 Memory Grant Feedback (Row Mode) to handle the memory allocation in Row
Mode. If a row mode query has operations that demand extra disk space, more
memory will be allocated for it, in the consecutive executions. While, if the query
uses less than 50% of the allocated memory, the memory grant will be reduced from
the following executions.
 Scalar UDF Inlining is adopted to bring larger performance gains. This mainly deals
with transforming the scalar UDFs into equivalent relational expressions, which are
"inlined" into calling the query.
 Table Variable Deferred Compilation which slightly differs from the interleaved
execution. This feature uses the actual cardinality of the table variable encountered
on the first compilation instead of a fixed
guess.

5. Always On Availability Groups

Always On availability groups is a disaster-recovery and a high-availability (HA) solution that

aims at providing an enterprise-level alternative to database mirroring. This feature was
initially introduced for SQL Server 2012, to increase the availability of a set of user databases
for an enterprise.
An availability group is designed to support the replicate environment for a set of user
databases called availability databases. An availability group can be either created for High
Availability (HA) or for read-scale.

The failure of an availability group happens at the level of an availability replica. Availability
database incurs, failover all together.

Features of Always On availability groups in SQL Server 2017

SQL Server 2017 introduces two sets of availability groups, differentiated on the basis of
their architecture.

Always On availability groups

This provides high availability, disaster recovery, and read-scale balance. The availability
groups here use the cluster manager in case of a cluster failover. In Linux, Pacemaker is used
for the same while, Windows uses a cluster manager.

Read-scale availability group

This architecture provides replicas only for the read-only workloads. They don't provide the
High Availability. There is no Cluster manager used in a Read-scale availability.

Every set of an availability database is hosted by an availability replica. The 2017 SQL Server
version provides only 2 types of the replica. They are the primary replica and the secondary
replica. An availability replica supports with redundancy only at the database level.

New Added Features of SQL Server 2019

Apart from the existing features of the SQL Server 2017, there are new improvements and
additions made to the SQL Server 2019.

 Unlike the older version, SQL Server 2019 increases the maximum number of replicas
from 2 to 5. Out of 5 replicas, 1 is the primary replica, while the other 4 are the
secondary replicas. You can configure these 5 replicas to handle the group failover.
 Secondary-to-primary replica connection redirection:
 This allows the client connections to be redirected to the primary replica, regardless of
the target specifications in the connection string. This connection provides the
connection redirection without a listener.
 Use secondary-to-primary replica connection redirection in the following cases:
1. The listener capability is absent in the cluster technology.
2. When the redirection becomes complex in a multi-subnet configuration.
 3. Scenarios of the read-scale out or disaster recovery, where the cluster type is
NONE.
 SQL Server 2019 provides for an ability to configure Always on Availability Groups
using Kubernetes as an orchestration layer in place of the Windows cluster failure.

Advantages of the new features of Always On availability groups in SQL Server

2019

 Increased number of availability replicas improve the availability during the disaster
recovery phase. For each availability database, a set of 4 secondary replicas and one
primary replica are available for recovery.
 The secondary to primary replica redirection improves the efficiency of database
management.
 Always On availability groups ensures that resource management is made efficiently
and the database availability is improved.

6. Machine Learning on Linux

Microsoft has always been keen on mixing up the data and the code. Microsoft SQL Server
has seen a transition of this trend from T-SQL, to Azure-focused U-SQL, which then extended
T-SQL with C# elements. SQL Server then added an embedded R support in 2016. In 2017,
this focus was extended to the addition of Python to the SQL Server. This has insanely
attracted the Machine Learning enthusiasts who are not even introduced to SQL Server!

Microsoft has always looked up to introduce new features that would make the SQL Servers
on Linux get in parity with the SQL Server on Windows. Some of the improvements in Machine
Learning, made in SQL Server 2019 on Linux, are discussed below.

Features of Machine Learning on Linux in SQL Server 2017

There are many advantages of having Python and R embedded in the SQL Server. Some of
them are listed below:

 Having Python embedded on SQL Server lets you take advantage of the Microsoft
scaling and performance features, by gaining direct access to the in-memory
database features and speeding up the OLAP requests.
 The code executed will be in the form of stored procedures. This enables the SQL
developers to just get the procedure and execute without having to worry about the
 code, while the data-scientists can take care of the written scripts. This ensures data
security.
 The dual support to the R and python in SQL server has been a logical move to
Microsoft. Since it runs both on the platform and on the cloud, SQL servers can work
with the traditional big data sources with all the data.

New Added Features of the Machine Learning in SQL Server 2019

 Besides R and Python, the new Java language extension is added to the SQL Server.
 AppContainers have replaced the local user accounts under SQL Restricted User
Group (SQLRUserGroup).
 The membership availability of SQLRUserGroup has changed. Instead of the multiple
local user accounts as in the previous version, SQL Server is just having a Launchpad
service account. All the processes of R, Python, and Java, now execute under the
Launchpad Service identity, isolated from the
AppContainers. [

7. SQL Server on Linux

 One of the biggest improvements of the SQL Server 2019 is the addition of the big
data clusters. Enhanced big data integration is one of the major areas of focus for
Microsoft SQL Server 2019.
 The big data clusters are supported on a number of technologies including SQL
Server on Linux in Docker containers, Apache Spark, Hadoop, and Kubernetes. The
big data clusters developed, allows the user to deploy the scalable cluster containers
on Kubernetes, which can read, write, and process the big data using the T-SQL.
 The big data cluster is made of the SQL Server and Spark Linux containers. Linux
containers use the Kubernetes for container management and orchestration. Multiple
Docker containers on a scalable group of nodes comprise the compute tier, on which
the queries run parallelly. Advanced analytics and Machine Learning is well
supported by Spark. Here, the big data clusters are managed by the master instance
of the SQL Server.
 Some of the other direct improvements for SQL Server on Linux in 2019 are the
extended support for transactional replication and distributed transactions.
 SQL Server 2019 on Linux instances can participate in the snapshot, merge, and
transactional replication topologies as a subscriber, publisher or distributor.
 The support for Microsoft Distributed Transaction Coordinator (MSDTC) allows for
distributed transactions on Linux instances of SQL Server. This was made possible
through the new Linux version of MSDTC that runs within the SQL Server process.
  SQL Server 2019 on Linux has better integration with Active Directory, which provides
functionalities like user authentication, replication, distributed queries, and AGs.
Furthermore, it also has OpenLDAP support for third-party AD providers. It also
provides for in-database machine learning.
 SQL Server 2019 on Linux has come up with a new Red Hat Enterprise Linux (RHEL)-
certified Docker container image: docker pull
mcr.microsoft.com/mssql/rhel/server:2019-CTP2.1
 Microsoft also has come up with a new master container registry. This registry is
meant to support existing catalogs like Docker Hub, Red Hat Container Catalog, and
Azure Marketplace.

8. Enhanced Security

Since SQL Server is directly dealing with the database management and procuring, the
security of the transactions and the data involved forms one of the most important
requirements.

Security for accessing the SQL servers is managed by certificates. The new security feature of
SQL Server 2019 includes Certificate management in SQL Server Configuration Manager (CTP
2.0). This certificate authenticates the secure access to SQL Server instances. The certificate
management is now dedicated to SQL Server Configuration Manager, hence easing other
tasks such as:

 Viewing and validating the certificates installed in the instance of SQL Server.
 Managing certificates that are nearing the expiration.
 Managing the certification deployment across machines, which are participating in
Always On Availability Groups.
 Managing the certificates deployment across machines, participating in failover cluster
instance.

Always Encrypted was used as the Microsoft solution for data security. However, there are
certain limitations to this method, where simple functions like mathematical operations
cannot be performed on the encrypted data.

To overcome this, a new technology called "Secure Enclaves" is adopted which allows simple
functions like the aggregate functions and LIKE queries to be executed on the Always
Encrypted data.
Microsoft SQL Server 2017 vs 2019

Below are a few distinctive features that differentiate the SQL Server 2017 version from SQL
Server 2019.

Topics SQL Server 2017 SQL Server 2019

Big Data A new feature of big data cluster incorporated to handle the
Was not included
clusters big data problems

Always Encrypted”
feature encodes
data. Encoded data
“Secure Enclaves” improvises over the previously encoded
cannot handle any
Security data by allowing the basic mathematical or relational
mathematical or
operations on encoded data.
relational
operations on
them.

Adaptive Joins in
Intelligent batch mode and
Along with previous version features, includes memory
Query memory feedback
feedback in row store mode and Scalar UDF Inlining.
Processing in batch mode
supported.

Resumable Online
Indexes Resumable Online Index Create
Index Rebuilding

Always On
5 replicas
availability 2 replicas
groups Secondary to primary index replica redirection
Doesn’t support the
Linux Supports OpenLDAP
OpenLDAP
Additional New Features in SQL Server 2019

Here are some other features enhancements.

Support to Persistent Memory (PMEM) Devices

SQL Server 2019 provides support to Persistent Memory (PMEM) devices. SQL Server
directly accesses the device, bypassing the storage stack of the operating system for
the files placed on the PMEM device.

Columnstore Index Enhancements

SQL Server 2019 also provides enhancements to columnstore index features such as
columnstore index maintenance, better metadata memory management, a low-memory
load path for columnstore tables, and improved performance for bulk loading to
columnstore indexes.

Resumable Online Index Creation

SQL Server 2019 also provides support for resumable online index creation similar to
resumable online index rebuilds in SQL Server 2017. Check out this tip
about resumable online index creation.

Up to Five Synchronous Replica Pairs for Availability Groups

We can now configure up to five synchronous replicas in an Availability Groups AG (one

primary and up to four secondary replicas) with automatic failover between these
replicas.

Enable High Availability Configurations for SQL Server Running in Containers

With SQL Server 2019, we can configure Always on Availability Groups using
Kubernetes as an orchestration layer.

Better Scale-out with Automatic Redirection of Connections Based on read/write

Intent

In SQL Server 2019, client applications can connect to any of the replicas of the
Availability Group. The connection redirects to the primary replica as per the AG
configuration and connection string.

SQL Data Discovery and Classification

We have explored this feature in SSMS 17.5. In SQL Server 2019, SQL Data discovery
and classification is integrated into the SQL Server engine with new metadata. This
enables us to ensure GDPR and other compliance needs for our databases.
Always Encrypted with Secure Enclaves

SQL Server 2019 introduces the secure enclave technology. A secure enclave extends
client applications, data trust to the server side. It secures the data from the malware
and privileged users.

Certificate Management Functionality in SQL Server Configuration Manager

In SQL Server 2019, certificate management is integrated into the SQL Server
Configuration Manager. We can view, validate the certificates being used in SQL Server
instance. We can view and validate certificates installed in a SQL Server instance. This
also provides detail about certification expiration dates. This feature helps to manage
certificates in a better way. We can also deploy certificates for AG instances starting
from the primary replica.

UTF-8 Support

SQL Server 2019 provides support for UTF-8 character encoding. We can now create a
char or varchar column to store UTF-8 data. This feature improves data compatibility
and performance improvements.

Vulnerability Assessment

We can use vulnerability assessment to track compliance of SQL Server instances and
Azure SQL Database instances with recognized security best practices. We can
implement using the reports shared by this tool. This provides easy to implement
security compliance such as GDPR.

Enhancement in SQL Graph

In SQL Server 2017 we have explored the graph databases, some of the tips for
reference are:

 Graph processing with SQL Server

 SQL Server 2017 Graph Database Example
 SQL Server 2019 provides enhancements to include match support with T-SQL MERGE
and edge constraints. We can now limit the type of nodes a given edge type can
connect. This helps users to enforce restrictions on the edge table. It also maintains data
integrity in their graph database.
 SQL Server 2019 also support MATCH predicates to specify graph relationships in a
single statement, instead of separate DML statements. This makes easy to query the
graph database.