Search This Blog

Showing posts with label SQL. Show all posts
Showing posts with label SQL. Show all posts

Thursday, February 2, 2017

LEN and DATALENGTH

LEN function is very commonly used function in t-sql. One thing to note in this function that is ignores the trailing blanks;
Declare @v VarChar(5)
Set @v = 'ati '
Select Len(@v)

Same goes for CHAR and NVARCHAR.
To overcome this, use and character at the end and minus 1 from the length J
Declare @v VarChar(5)
Set @v = 'ati '
Set @v = @v + '?'
Select Len(@v)-1

Some people might use DATALENGTH function;

Declare @v VarChar(5)
Set @v = 'ati '
Select  DATALENGTH(@v)

But be alert as DATALENGTH function counts the bytes, not the length. If you change the data type of @v from varchar to nvarchar, the result will change;

Declare @v NVarChar(5)
Set @v = 'ati '
Select  DATALENGTH(@v)

Thanks for reading.




XACT_ABORT V/S TRY .. CATCH

Few days back we witnessed an issue that the SQL Server was running slow. We investigated and soon found out that there were uncommitted transactions on the server. We re-confirmed those transactions from the developers and resolved the issue. Someone tossed the idea to apply SET  XACT_ABORT ON in all SPs so that this situation may not occur again. That made me to write this article so that I can explain the pros and cons of SET XACT_ABORT ON option.
SET XACT_ABORT ON
As many of you already know that this options just takes the whole batch a one transaction. If any of the statement in the batch fails, the transaction of the whole batch is rolled back. Commits in case of success. Simple. But not that simple. You cannot control the flow of transaction with if XACT_ABORT is ON. TRY .. CATCH will not work. Even if you have the error log is implemented in CATCH block, it will not be saved as it will also get rolled backed. So, DB end error log is gone. In some cases you may need to continue with the rest of your code in case of an error, for example return a dataset. You cannot do this as well. The plus point you will never have an open transaction on your server. All transactions will either commit or rollback.
Another way to handle an open transaction issue is to apply TRY .. CATCH in all SPs. Ok, I know. There are few limitations of TRY .. CATCH. TRY .. CATCH do not catch the schema related errors. It will return an error without hitting CATCH block where you have rolled back opened transaction in case of a schema error (table does not exists, etc ). Check the code below;
       BEGIN TRY 
              -- Table does not exist; object name resolution 
              -- error not caught. 
              begin Tran
              SELECT * FROM NonexistingTable;
              commit 
       END TRY 
       BEGIN CATCH 
              rollback
              SELECT  
                     ERROR_NUMBER() AS ErrorNumber 
                 ,ERROR_MESSAGE() AS ErrorMessage; 
       END CATCH 
      
       GO    
       Select @@TRANCOUNT
      
Execute the above statements. You will end up with an opened transaction.
You can also try this in a stored procedure. You will still end up in an open transaction.
Bow add SET XACT_ABORT ON;
       SET XACT_ABORT ON
       BEGIN TRY 
              -- Table does not exist; object name resolution 
              -- error not caught. 
              begin Tran
              SELECT * FROM NonexistingTable;
              commit 
       END TRY 
       BEGIN CATCH 
              rollback
              SELECT  
                     ERROR_NUMBER() AS ErrorNumber 
                 ,ERROR_MESSAGE() AS ErrorMessage; 
       END CATCH 
      
       GO    
       Select @@TRANCOUNT

Your transaction will be rolled back and you will find no open transaction.
CONCLUSION
XACT_ABORT ON solves the open transaction issue but at the cost of logging implemented in CATCH block (if you have any).
But “NonexistingTable” type of issues should not be in the production environment. Like why will I deploy a SP using a table that does not exists? Strange thing for me. You should have QA in place and your deployment scripts should be verified before running in the production environment.

 You decide.

Wednesday, February 1, 2017

SET DATEFIRST and SET LANGUAGE

This is a tricky part. SQL Server have language settings at the following levels;
1.       Server
2.       User Login
3.       Session
You need to be very clear for the language settings of the user / login as it may impact the date functions;
DECLARE @Today DATETIME; 
SET @Today = getdate(); 
 
SET LANGUAGE Italian; 

SELECT DATENAME(month, @Today) AS 'Month Name'; 
Select @@DATEFIRST
 
SET LANGUAGE us_English;
SELECT DATENAME(month, @Today) AS 'Month Name' ; 
Select @@DATEFIRST


SET LANGUAGE British;
SELECT DATENAME(month, @Today) AS 'Month Name' ; 
Select @@DATEFIRST

SET LANGUAGE us_English;

GO 

As you can see from the above code, I have set the language back to US English (English in SQL Server language drop downs) as I like to have it that way. You also see that @@DATEFIRST function returns different value for different languages. This value is set on the basis of;
1.       Language selected
2.       SET DATEFIRST option

You can change the language and still keep the First date of week same as in US English;
SET LANGUAGE us_English;
Select @@DATEFIRST

SET LANGUAGE British;
Select @@DATEFIRST
SET DATEFIRST 7
Select @@DATEFIRST

As you can see, US_English sets the @@DateFirst to 7 and British sets it to 1. But we have used SET DATEFIRST to change the language to British but kept @@DATEFIRST  to 7 using SET DATEFIRST option.

This is very important when you are working multilingual databases. Where ever you are using day of week values (in DatePart, etc), you should keep this change in mind as well. Either use one language throughout your database for Server, users and session. But if there might be any change to get the benefit of SQL Server multilingual support, you should use SET DATEFIRST into account when programming to keep all the calculations and checks aligned.

Monday, January 30, 2017

xp_fileexist And its Alternate

xp_fileexists is a very useful undocumented stored procedure of SQL Server.
The usage of xp_fileexist is as follow;
Exec xp_fileexist “E:\abc.txt”
The values returned are



You can also use OUTPUT parameter to get the value of “File Exists” column as below;

Declare @vFileExists int
exec master.dbo.xp_fileexist 'E:\abc.txt', @vFileExists OUTPUT
Select @vFileExists

If you want to save all three returned values, then you will have to go throu Temp Table approach;

Declare @vFileExists Table (FileExists int, FileDir int, ParentDirExists int)

insert into @vFileExists
       exec master.dbo.xp_fileexist 'E:\abc.txt'
Select * from @vFileExists




Sometimes xp_fileexist behaves abnormally. Your file is there and it does not check the file and returns 0 in FileExists column. This is an extended SP so its behavior may also change with different versions of SQL Server.
If you are facing the same problem, change you code with xp_cmdshell “dir” approach;

Declare @vExistsPath nvarchar(100)
Declare @files Table ([FileName] nvarchar(100))
Set @vExistsPath = ''
Set @vExistsPath = 'E:\abc.txt'
Set @vExistsPath = 'dir ' + @vExistsPath + ' /b'
Insert into @files EXEC xp_cmdshell @vExistsPath
Select * from @files
if Exists(Select 1 from @files where [FileName] = 'abc.txt' And [FileName] is Not Null)
begin
       Select 1
end
else
begin
       Select 0
end

The “/b” switch returns only filename with extension as a result of “dir” command





Tuesday, March 18, 2014

Optimizing SQL Azure Database

1. Dynamic Management Views
SQL Azure provides a few handy system views called dynamic management views (DMVs) that are also available in SQL Server. SQL Azure exposes a subset of the DMVs, but all those related to query execution are available.

sys.dmexecconnections 
sys.dmexecqueryplan 
sys.dmexecquerystats 
sys.dmexecrequests 
sys.dmexecsessions 
sys.dmexecsqltext 
sys.dmexectextquery_plan

2. Connection pooling
Connection pooling is an important consideration for performance. Although this statement is generally accurate in the world of database programming, it becomes critical for SQL Azure. A poorly designed application may create too many connection requests, which can end up flooding SQL Azure. If too many connection requests are established, your connections will be throttled, meaning that you can no longer connect for a period of time.

3. Indexes
Creating the right indexes can be complex; it can take a long time to fully understand indexing and fine-tune database queries. One of the most important things to remember with indexing is that its primary purpose is to help SQL Azure find the data it needs quickly.

Indexes are like smaller tables that contain a subset of the primary table. The tradeoff is that indexes consume space and must be maintained by the SQL Azure engine as the primary data changes, which can impact performance under certain scenarios.

Remember, you must have a clustered index on each table in SQL Azure Database. Windows Azure SQL Database does not support tables without clustered indexes. A table must have a clustered index. If a table is created without a clustered constraint, a clustered index must be created before an insert operation is allowed on the table.

4. Indexed views 
Indexed views are an excellent alternative when you absolutely need to JOIN data, and traditional indexing doesn't yield the performance you're looking for. Indexed views behave like tables; the data covered is materialized to disk so it can be retrieved quickly. Before jumping on indexed views, understand that they have certain limitations and that due to their nature, you may incur a performance hit through the usual Insert, Delete, and Update statements.

5. Stored procedures
You've seen various ways to tune your statements and improve execution plans. However, keep in mind that you also have stored procedures at your disposal. Stored procedures can give you an edge if you need to execute logic that requires a large volume of data. Because you know that returning lots of data turns into a performance problem in SQL Azure, you can place the business logic that needs the data in a stored procedure, and have the procedure return a status code. Because you aren't charged for CPU time, this becomes an affordable option.

Stored procedures can also be an interesting security tool, allowing you proxy the calls to underlying tables through a procedure and never allowing direct access to the tables.

There are a few restrictions in SQL Azure Stored Procedures. You cannot use the following in CREATE PROCEDURE statement;

  • FOR REPLICATION option
  • EXTERNAL NAME ( external methods)
  • ENCRYPTION option

Also, following XML commands doe not work in SQL Azure;
  • sp_xml_preparedocument
  • sp_xml_removedocument
  • OpenXML
We have the workaround to tackle with these commands but that is not in the scope of this article.

NOTE: Reference taken from http://programming4.us/

Tuesday, November 19, 2013

Sequence Object in SQL Server 2012

MS SQL Server has introduced a new feature of Sequence Objects in SQL Server 2012. In the prior versions, IDENTITY was used to generate sequence numbers. The limitation of IDENTITY was that it was related to one table. The scope of Sequence Objects is database. So, if you want to add a same value to two tables, you can use this feature. Also, you can generate the next sequence number before going into the transaction. The IDENTITY is incremented in the transaction at the time of INSERT. Another plus point of Sequence Object is that you can define the maximum value for the object and re-initialize the counter of the object without any extra programming effort.

Here is the code of creating teh new sequence object in SQL Server 2012 Management Studio;

CREATE SEQUENCE dbo.NextCustomerID
    as BIGINT
    START WITH 1
    INCREMENT BY 1;
GO

This will add a new sequence object. You can see this object in Object Explorer under your database in Programming --> Sequence folder.


Well, it not over yet. The query used to create this object creates the object with the default values. You can set many parameters. If you right click the newly created object and click properties, this will open the property window for this object. Here you can set Minimum value, Maximum Value, Cycle and Cache.


Minimum and Maximum are the minimum and maximum values for the object. Cycle check box states that the value will re-initialize when the maximum value is reached. You can also set the re-start value (From where it will start after reaching the maximum value). Cache option tells the object how many values it will have remaining to avoid extra Disk I/Os during the increment process. For example, if cache size is set to 25, it will  not generate 25 numbers in it memory. It will only keep the current value and the remaining values. If 3 IDs are generated for the object with cache of 25, it will have 4 and 22 in its memory. For more information on cache mechanism you can Click Here and check the Cache Management section.

Ok, here is the code for generating the sequence number from the object;

-- Get the next available ID
DECLARE @CustID BIGINT;
SELECT @CustID = NEXT VALUE FOR dbo.NextCustomerID;
SELECT @CustID as 'Customer ID';

Please remember that the sequence object is not transaction specific. Once you generate the sequence with the NEXT VALUE, it is consumed weather the transaction is committed and rolled back. This is similar to the IDENTITY. Hence, the sequence objects also may have gaps in between as we have in IDENTITY columns.

In case you want to restart the sequence, you can use;

ALTER SEQUENCE dbo.NextCustomerID
RESTART

You can use the OVER ORDER BY clause with NEXT VALUE FOR for generating the next sequence values;

SELECT NEXT VALUE FOR dbo.NextCustomerID OVER (ORDER BY RNO) AS ListNumber,
    Name
FROM tblTest ;

You can check books online for more information.

There are also the limitations of the Sequence Objects which I will cover in the separate topic and try to explain with examples.

Monday, November 18, 2013

Columnstore Indexes

This is the new feature added in SQL Server 2012 to overcome query performance and index data compression issues. The data stored in the columnstore index is pretty much compressed and very optimized. In SQL Srever 2012, if you have a columnstore index, you cannot perform the DML operations on the table i.e. you cannot insert / update or delete any record. The purpose of this index is for Data warehousing. You have to devise the logic in order to populate the data, for example table partitioning. But this issue is resolved in SQL Server 2014. But there, you will have to create a CLUSTERED COLUMNSTORE INDEX.

You can add this index simply by right clicking the Index folder under the table on which you want to add the column store index;


On clicking the option, a screen will open asking to add the column(s) on which you want to create a column store index (FirstName, LastName).


Leave rest of the properties to default. Click OK button to add the index.

Now, question is how the data will be stored? We have two types of data storge in sql server, RowStore and Column Store. Assume you have a tabel as

RNO       NAME          FNAME
1              Atif                Sheikh
2              Asif                Ahmed
3             Imran              Khan

In RowStore index, the data will be saved as;

Atif,Sheikh
Asif,Ahmed
Imran,Khan

The above data will be saved as a row in the 8k page.

And this will go on until the 8k page is complete.

In case of Columnstore index, the data will be stored as a segment of its respective column. Each segment have only one column. This goes for all the columns in the index. A Column can span on multiple segments and a segment may have multiple data pages. As for above example the data will be stored as;

Col1               Col2

Atif                  Sheikh
Asif                 Ahmed
Imran              Khan

These columns are just like pointers in the 8k page of the index. One page can have multiple columns

Now if you will query with NAME or FNAME in your WHERE condition, the data search and retrieval will be very fast as each condition will have to check one single column. You can check this with Statistics and Execution plans on you tables.

Wednesday, November 13, 2013

Paging in SQL Server 2012

As we all know that MS has given us the FETCH NEXT clause in the query to apply paging. Let me explain this feature by adding a temp table of 1 million records.

with wcte as (
Select Row_number()over (order by a.name) as RNO
from sys.columns a, sys.columns b, sys.columns c
) Select rno , 'Name: ' + CAST(rno as varchar(10))  as Name
into tblTest
from wcte where RNO <= 1000000



Add primary key on RNO column through the designer. This will add a clustered index on this column.

Now, lets write the query for the SQL Server 2005 / 2008 / 2008 R2 using CTE;

;With wcte as (
Select Row_Number() over (Order by Rno) Rno2,* from tblTest WHERE Name is Not Null
) Select * from wcte where rno2 > 899979 and rno2 < 900000
order by rno



 If you look at the execution plan of this query, it will be like this;


Now lets write the query for SQL Server 2012 paging;

Declare @vPageNo int
Declare @vOffset int
Declare @vPageSize int

Set @vPageNo = 45000
Set @vPageSize = 20
Set @vOffset = (@vPageNo-1) * @vPageSize

SELECT *
FROM tblTest
WHERE Name is Not Null
ORDER BY rno
   OFFSET @vOffset ROWS
   FETCH NEXT @vPageSize ROWS ONLY


I have added the variables for passing OFFSET and PAGESIZE. This gives an additional hint for passing these from the front end as Input parameters of stored procedure.

The execution plan for this query is;

Now, it is very clear that the paging query of SQL Server 2012 performed very limited functions and performed 100% index scan. The SQL Server 2008 query performed many additional functions which makes it a bit smaller with my 2 column table.

Results might be different if you apply the same scenario directly on the clustered indexed column (RNO of tbltest in my case) instead of applying Row_Number() function. This is not the real time case as you never have IDs in sequence in you db tables as sequences break when you delete or migrate. More importantly, you apply paging on queries with WHERE clause. And you cannot use ID columns for paging in any case with WHERE Clause.

Monday, November 11, 2013

OUTPUT CLAUSE


Returns information from, or expressions based on, each row affected by an INSERT, UPDATE, DELETE, or MERGE statement. These results can be returned to the processing application for use in such things as confirmation messages, archiving, and other such application requirements. The results can also be inserted into a table or table variable. Additionally, you can capture the results of an OUTPUT clause in a nested INSERT, UPDATE, DELETE, or MERGE statement, and insert those results into a target table or view.
OUTPUT clause can generate table variable, a permanent table, or temporary table.

Using an OUTPUT Clause in an INSERT Statement

When you insert data into a table, you can use the OUTPUT clause to return a copy of the data that’s been inserted into the table. The OUTPUT clause takes two basic forms: OUTPUT and OUTPUT INTO. Use the OUTPUT form if you want to return the data to the calling application. Use the OUTPUT INTO form if you want to return the data to a table or a table variable.


Create Table MyTable (ID int, name varchar(100),createdAt datetime)

-- declare @InsertOutput2 table variable
DECLARE @InsertOutput2 table
(
  ID int,
  name nvarchar(50),
  CreatedAt datetime
);

 -- insert new row into table in your database
INSERT INTO MyTable
OUTPUT
    INSERTED.ID,
    INSERTED.Name,
    INSERTED.CreatedAt
  INTO @InsertOutput2
VALUES(111, 'Some text', GETDATE());


-- view inserted row in table
SELECT * FROM MyTable;
-- view output row in @InsertOutput2 variable
SELECT * FROM @InsertOutput2;

drop table MyTable

Using an OUTPUT Clause in an UPDATE Statement

Create Table MyTable (ID int, name varchar(100),createdAt datetime)

-- declare @InsertOutput2 table variable
DECLARE @UpdateOutput1 table
(
  ID int, 
  name nvarchar(50),
  Oldname nvarchar(50),
  CreatedAt datetime,
  UpdatedAt datetime
);

 -- insert new row into table in your database
INSERT INTO MyTable
VALUES(111, 'Some text', GETDATE());


UPDATE MyTable
SET  
  name = 'Changed...'
OUTPUT   
    INSERTED.ID,
    INSERTED.Name,
    DELETED.Name,
    DELETED.CreatedAt,
    getdate()
  INTO @UpdateOutput1
WHERE Name = 'Some text';



-- view inserted row in table
SELECT * FROM MyTable;
-- view output row in @InsertOutput2 variable
SELECT * FROM @UpdateOutput1;

drop table MyTable

In the same manner you can use it with DELETE statement. It uises the same INSERTED and DELETED memory tables.

Using an OUTPUT Clause in an MERGE Statement

When working in SQL Server 2008, you can add an OUTPUT clause to a MERGE statement. The process is similar to adding the clause to an UPDATE statement; you use both the INSERTED and DELETED column prefixes.
Let’s look at an example that demonstrates how this work. First, however, we must create a second table to support the MERGE statement. The following script creates the Book2 table and populates it with two rows:



CREATE TABLE dbo.Books2
(
  BookID int NOT NULL PRIMARY KEY,
  BookTitle nvarchar(50) NOT NULL,
  ModifiedDate datetime NOT NULL
);


CREATE TABLE dbo.Books
(
  BookID int NOT NULL PRIMARY KEY,
  BookTitle nvarchar(50) NOT NULL,
  ModifiedDate datetime NOT NULL
);



INSERT INTO Books2
     VALUES(101, '100 Years of Solitude', GETDATE());



INSERT INTO Books2
     VALUES(102, 'Pride & Prejudice', GETDATE());

--Once we’ve created the Books2 table, we can try a MERGE statement. In the following example, I declare the @MergeOutput1 variable, merge data from the Books table into the Books2 table, and view the results:
-- declare @MergeOutput1 table variable

DECLARE @MergeOutput1 table
(
  ActionType nvarchar(10),
  BookID int,
  OldBookTitle nvarchar(50),
  NewBookTitle nvarchar(50),
  ModifiedDate datetime
);


INSERT INTO Books
     VALUES(101, 'The Great Gatsby', GETDATE());


-- use MERGE statement to perform update on Book2
MERGE Books2 AS b2
USING Books AS b1
ON (b2.BookID = b1.BookID)
WHEN MATCHED
THEN UPDATE
SET b2.BookTitle = b1.BookTitle
OUTPUT
    $action,
    INSERTED.BookID,
    DELETED.BookTitle,
    INSERTED.BookTitle,
    INSERTED.ModifiedDate
  INTO @MergeOutput1;


-- view Books table
SELECT * FROM Books;

-- view updated rows in Books2 table
SELECT * FROM Books2;

-- view output rows in @MergeOutput1 variable
SELECT * FROM @MergeOutput1;



drop table Books
drop table Books2


$action Gives us the type of action that is performed by Merge statement per record.