SQL Server versions
Columnstore
Columnstore indexes - Overview
Columnstore indexes - what's new
Columnstore indexes - Design guidance
Columnstore Index Design Guidelines
Columnstore indexes - Query performance
What are Columnstore Indexes?
Indexes
- Overlapping indexes
- Kendra Little - Everything About Your Indexes
SQL Server Agent Jobs
- Get my own JOB_ID
- Copy to clipboardDECLARE @name NVARCHAR(128) select @name = name from msdb.dbo.sysjobs where job_id = $(ESCAPE_SQUOTE(JOBID)) PRINT @name
- If you "parse" the code, ignore the error: "Incorrect syntax near 'ESCAPE_SQUOTE'."
-
- Using Tokens in Job Steps
- Set Output file name for all jobs
- Copy to clipboard_ UPDATE a SET a.output_file_name = b.name + '.txt' FROM msdb.dbo.sysjobsteps a inner join msdb.dbo.sysjobs b ON a.job_id = b.job_id
-
- Update to include step output in history
- Copy to clipboard_ SELECT j.name, flags | 32 AS NewFlag, js.* -- UPDATE js SET flags = flags | 32 FROM msdb.dbo.sysjobsteps js INNER JOIN msdb.dbo.sysjobs j ON j.job_id = js.job_id WHERE flags & 32 = 0 AND subsystem IN ('CmdExec', 'SSIS') SELECT j.name, flags | 4 AS NewFlag, js.* -- UPDATE js SET flags = flags | 4 FROM msdb.dbo.sysjobsteps js INNER JOIN msdb.dbo.sysjobs j ON j.job_id = js.job_id WHERE flags & 4 = 0 AND subsystem = 'TSQL'
- http://www.nuronconsulting.com/TemplateScripts_msdbSet_Include_step_in_output_history_sql.aspx
- msdb.dbo.sysjobsteps.flags
- OR together as bits in a field
- subsystem IN ('CmdExec', 'SSIS', 'PowerShell')
- none = 0
- Append output to existing file = 2
- log to table = 8
- append to table = 16 — mutually exclusive with log to table
- Include step output in history = 32
- subsystem = 'TSQL'
- none = 0
- Append output to existing file = 2
- include step output in history = 4
- log to table = 8
- append to table = 16 — mutually exclusive with log to table
VLFs
- SQL Script to list VLF counts for all online databases
- VLFs created for log file growths
- chunks less than 64MB and up to 64MB = 4 VLFs
- chunks larger than 64MB and up to 1GB = 8 VLFs
- chunks larger than 1GB = 16 VLFs
- Source: http://www.sqlskills.com/blogs/kimberly/transaction-log-vlfs-too-many-or-too-few/
- Don't grow in multiples of 4096 MB because of bug
Finding equivalent elements in SQL (regrouping)
Logging SSPI failed logins on computers on a doman
- Must be on a domain to enable logging for the netlogon service
- Run nltest to turn on logging. Included in Win 2008 +, should be on resource disk for Win 2003 +
- Basic info to turn on: https://support.microsoft.com/en-us/kb/109626
- Detailed info on nltest: http://blogs.technet.com/b/askpfeplat/archive/2013/01/28/quick-reference-troubleshooting-netlogon-error-codes.aspx
- Blog on using nltest that revealed problem was domain controller and not SQL Server: http://www.allenkinsel.com/archive/2010/06/sql-server-and-sspi-handshake-failed-error-hell/
SQL Server TLS 1.0
- Seems that SQL Server, all versions, require TLS 1.0 to be turned on
- http://dba.stackexchange.com/questions/82207/sql-server-service-wont-start-after-disabling-tls-1-0-and-ssl-3-0-on-windows?atw=1
- https://social.msdn.microsoft.com/Forums/sqlserver/en-US/5f543d1f-9483-49bb-9c65-999f287eaefa/sql-server-service-wont-start-after-disabling-tls-10-and-ssl-30-on-windows?forum=sqlsecurity
- http://dba.stackexchange.com/questions/93127/sql-server-service-won-t-start-after-disabling-tls-1-0-and-ssl-3-0
- http://www.experts-exchange.com/questions/28652578/Disabling-TLS-1-0-and-SSL-3-0-causes-SQL-2014-services-to-not-start.html
- http://msdn.developer-works.com/article/12693133/Enable+TLS+1.2+for+SQL+SSL+connections%3F
- http://stackoverflow.com/questions/27222142/ms-sql-server-fails-to-start-after-disabling-ssl-v3-protocol
Ports to open
- TCP 135 - SQL Transact Debugger/RPC
- TCP 1433 - SQL Server Default Instance
- TCP 1434 - SQL Server Admin Connection
- TCP 2382 - SQL Server Browser
- TCP 2383 - SQL Analysis Services
- TCP 4022 - SQL Service Broker
- UDP 1434 - SQL Server Browser Multicast Response
- Program Name - C:\Program Files\Microsoft SQL Server\MSSQLxxxx\MSSQL\Binn\sqlservr.exe
List of permissions for Service account
SMTP - DBMail
- Use Telnet to Test SMTP Communication
- Status of E-Mail Messages Sent With Database Mail
Errors
- File ID 1 of database ID 7 cannot be shrunk as it is either being shrunk by another process or is empty
- Grow file by 1MB
- Should be able to shrink after that
Instant File Initialization
- Only for data files, not on log files
- Testing if turned on, using trace flags
- Copy to clipboardDBCC TRACEON(3004,3605,-1) GO CREATE DATABASE TestFileZero GO EXEC sp_readerrorlog GO DROP DATABASE TestFileZero GO DBCC TRACEOFF(3004,3605,-1)
- Will always see zeroing out for log file: Zeroing completed on C:\Program Files\Microsoft SQL Server\MSSQL10.SQL08\MSSQL\DATA\TestFileZero_log.LDF
- If no similar message for mdf file, then it is turned on. If message for mdf exists, then it is not turned on.
- Source: http://blogs.msdn.com/b/sql_pfe_blog/archive/2009/12/23/how-and-why-to-enable-instant-file-initialization.aspx
-
Lock pages in memory
- Need to add the proper rights using the User / Groups settings, and need to set a trace flag
- ***UPDATE ACTUAL DIRECTIONS***
Adding users to non-domain Windows (home) versions
- Some versions of Windows do not support User Groups, such as Windows 7 Home or Windows 8 Standard
- Need to add SQL Server users individually - can't create or assign user groups
- https://books.google.com/books?id=JFgnCgAAQBAJ&pg=PA94&lpg=PA94&dq=sql+server+windows+8+home+add+user+group&source=bl&ots=8D4X_BZFsq&sig=5s8t1LAnWPWI4s78SzI1HIt9v_c&hl=en&sa=X&ved=0CFAQ6AEwB2oVChMI-9acirxwIVCaGACh3nYQ7B#v=onepage&q=sql%20server%20windows%208%20home%20add%20user%20group&f=false
SYS.DATABASES collations issue between SQL_Latin1_General_CP1_CI_AS and Latin1_General_CI_AI
- Some internal columns are statically set for internal operations
- What's up with the collation of some columns in sys.databases?
- Examples of changing collation
- Cannot resolve the collation conflict between “SQL_Latin1_General_CP1_CI_AS†and “Latin1_General_CI_AI†in the equal to operation
Sample Databases
- Adventure Works 2014
- Adventure Works for SQL Server 2012
Workload generators
- SQLStress
- Allows workloads to be created according to a workload profile that can be specified in percentages of insert, update, select and delete operations
- http://www.sqlstress.com/Overview.aspx
- SQL Load Generator
- Run multiple concurrent queries against SQL Server. The user can choose the number of concurrent queries to run, provide different queries
- https://sqlloadgenerator.codeplex.com/
- HammerDB
- Database Load Testing and Benchmarking Tool
- http://sourceforge.net/projects/hammerora/
- Kendra Little: Simulating OLTP Queries: How to Run Load Tests Against SQL Server with HammerDB
SQL Server 2000 SMTP mail
- Use CDO for Microsoft Windows 2000 (CDOSYS) (sp_OACreate, sp_OASetProperty, ...)
- Gert Draper's xp_smtp_sendmail
Disk IO Tests
- General Info
- SQLIOSim
- Tool performs reliability and integrity tests on disk subsystems. These tests simulate read, write, checkpoint, backup, sort, and read-ahead activities for Microsoft SQL Server. However, if you have to perform benchmark tests and to determine I/O capacity of the storage system, you should use the SQLIO tool
- Installed starting SQL Server 2008
- C:\Program Files\Microsoft SQL Server\MSSQLXX.XX\MSSQL\Binn\SQLIOSIM.EXE
- How to use the SQLIOSim utility to simulate SQL Server activity on a disk subsystem
- Examples
- https://www.simple-talk.com/sql/database-administration/the-sql-server-sqliosim-utility/
- Copy to clipboardsqliosim -dir c:\datafiles\sim -log c:\datafiles\sim\simlog.xml -d 60 -size 2
- -dir = folder to put both data and log files
- -l = file name for output report
- -d = seconds to run test
- -size = MB of data file to create (Not sure how size of log file is calculated)
- Download
- By default, the logs go to C:\Windows\System32\ if a path is not specified
- SQLIO
- Determines the I/O capacity of a given configuration
- http://www.microsoft.com/en-us/download/details.aspx?id=20163
- Crystal Disk Mark
- Quick, easy disk tests
- Has been rewritten to use DiskSpd
- http://crystalmark.info/software/CrystalDiskMark/index-e.html
- Microsoft DiskSpd
- Generate a wide variety of disk request patterns than Crystal Disk Mark. Synthetic storage subsystem testing when you want a greater level of control than that available in CrystalDiskMark.
- Download
- Good guide
Disk Layout
- Align Partitions
- Disk Partition Alignment Best Practices for SQL Server
- https://technet.microsoft.com/en-us/library/Dd758814(v=SQL.100).aspx
Profile Traces
- Write your own
- Free
- Using a desktop application written in C#, .NET 4.
- ExpressProfiler
- Free
- aka SQLExpress Profiler
- xSQL Profiler
- Free for 1 instance, $200 for multiple monitoring
- Server Side Trace
- Trace Flag to Error Log
- Free
- Requires restarting the SQL Server to turn on, but not to turn off
- Flags 4032, then 3605
SSIS
- Stop Error Propagation
- Script Task
- Using variables
- Copy to clipboardDts.Variables("fileName").Value
-
- Display Message
- Copy to clipboardMsgBox("First line" & vbCrLf & _ "Second line")
-
- Check is file exists
- Copy to clipboardIf (File.Exists(CStr(Dts.Variables("filePath").Value))) Then Dts.TaskResult = Dts.Results.Success Else Dts.TaskResult = Dts.Results.Failure End If
-
- Using variables
- STFP, ZIP, PHP Excryption
- SFTP, encrypt or compress data files in SSIS using custom components
- http://www.sqlservercentral.com/articles/Integration+Services+(SSIS)/106616/
- Web Site: http://ssisextensions.codeplex.com/releases/view/101949
- 2008 R2 Download Link: http://ssisextensions.codeplex.com/downloads/get/623584
- 2012 Download Link: http://ssisextensions.codeplex.com/downloads/get/616818
- Display Datetime in Expressions
- Copy to clipboard(DT_STR, 4, 1252) DATEPART("yyyy", GETDATE()) + "-" + RIGHT("0" + (DT_STR, 2, 1252) DATEPART("mm", GETDATE()), 2) + "-" + RIGHT("0" + (DT_STR, 2, 1252) DATEPART("dd", GETDATE()), 2) + "_" + RIGHT("0" + (DT_STR, 2, 1252) DATEPART("hh", GETDATE()), 2) + "-" + RIGHT("0" + (DT_STR, 2, 1252) DATEPART("mi", GETDATE()), 2) + "-" + RIGHT("0" + (DT_STR, 2, 1252) DATEPART("ss", GETDATE()), 2)
-
SSRS
- Dynamic Embedded Data Source
- Requires using embedded data source, not shared data source
- http://www.keepitsimpleandfast.com/2012/08/how-to-use-dynamic-data-sources-in-your.html
- Dynamic Shared Data Source
- Many more steps
- Uses RSExplorer++ tool
- http://www.codeproject.com/Articles/355461/Dynamically-Pointing-to-Shared-Data-Sources-on-SQL
SQL Server Data Tools (SSDT) in 2014
- Now separated out - SQL Server from Visual Studio
- Download from: https://msdn.microsoft.com/en-us/library/mt204009.aspx
- From the section at the bottom: "Download SQL Server Data Tools Business Intelligence (RTM)"
- Get either version
- It includes the Visual Studio Shell
- About 1GB download files
- 2013: https://www.microsoft.com/en-us/download/details.aspx?id=42313
- 2012: https://www.microsoft.com/en-us/download/details.aspx?id=36843
Data and Plan Cache
- What Data is in SQL Server's Memory?
- Plan Cache and Data Cache in Memory
- Finding what queries in the plan cache use a specific index
- (and) What’s Wrong about Page Life Expectancy >= 300?
- sys.dm_os_buffer_descriptors
Currently Running Processes
- Finding Locking & Blocking……..sys.dm_tran_locks
- Find Currently executing Queries, Blocking, Waits, Statement, Procedure, CPU
Performance Tuning and Monitoring
SSMS Custom Reports
- Creating custom reports, and the variables that get passed in
SSMS Settings
- Line Split copying from SSMS Results to Excel
- "In SSMS before 2012, the carriage return (\n\r) wasn't taken into consideration when copying data from SQL Server to Excel. This behaves in a different way in SQL Server 2012, as it preserves the carriage return (\n\r) and therefore splits the row into multiple rows in Excel when pasting."
- \n = CHAR(13)
- \r = CHAR(10)
- In SSMS 2012+ , this is fixed by adding a new option Retain CR\LF on copy or save under the Tools -> Options... menu, Query Results -> SQL Server -> Results to Grid.
Add-Ons
- SSMS Tools Pack
- add-in for Microsoft SQL Server Management Studio (SSMS) 2005, 2008, 2008 R2, 2012
- Execution Plan Analyzer
- SQL Snippets
- Window Connection Coloring
- Window Content History, Query Execution History, Current Window History and Tab Sessions
- Format SQL
- Search Table, View or Database Data
- Run one script on multiple databases
- Copy execution plan bitmaps to clipboard or file
- Search Results in Grid Mode
- Generate Insert statements from resultsets, tables or database
- Regions and Debug sections
- Running custom scripts from Object Explorer
- CRUD stored procedure generation
- New query template
- General options
- http://www.ssmstoolspack.com/
- SQL#
- SQLCLR package with new functions not in SQL
- Free and paid version
- Most of the free ones are easy enough to code in SQL, so haven't used it
- http://www.sqlsharp.com/
- SQL Query Stress
- Free tool
- Assist with performance stress testing of T-SQL queries and routines
- Automatically collects metrics to help determine how queries perform under load, and what kind of resource strain they put on server
- http://www.datamanipulation.net/sqlquerystress/
- SQL Co-Pilot
- Ready made SSMS Custom Reports
- Free and paid versions available
- http://www.sqlcopilot.com/getting-started.html
SQL Tools and Plugins
- SQLRanger
- Might be neat, but requires .Net 4.5 and installs as a ClickOnce Application, so hard to get it to run under different users
- Setting Windows Credentials does not change user context when connecting to other domains
- Run from the client computer, doesn't need to install on server
- http://www.sqlranger.com/
- Top 10 Free SQL Server Tools
- Qure Profiler
- Realtime profiling solution for analyzing and comparing SQL Server trace files
- Ability to compare database workloads before-and-after changes
- Free part of their full suite
- http://www.dbsophic.com/qure-analyzer.html
SSMSBoost
DataGrip
SSMS Tools Pack
dbForge SQL Complete
- https://blog.devart.com/ssmsboost-addin-for-ssms-vs-sql-complete.html
- Free version is very functional
- This has IntelliSense for Synapse
Business Intelligence
- __31 Open Source and Free Business Intelligence Solutions
CSV Splitter Functions
- http://www.sqlservercentral.com/articles/Tally+Table/72993/
- http://sqlblog.com/blogs/adam_machanic/archive/2009/04/28/sqlclr-string-splitting-part-2-even-faster-even-more-scalable.aspx
- http://www.sqlservercentral.com/Forums/Topic1101315-203-14.aspx#bm1102959
Schema Binding
- http://blogs.msdn.com/b/sqlprogrammability/archive/2006/05/12/596424.aspx
- http://beyondrelational.com/modules/2/blogs/76/posts/13754/the-benefits-of-schemabinding.aspx