Microsoft Bob https://docs.microsoft.com/archive/blogs/microsoftbob/feed.xml NOTE: The views expressed on this blog are my own and may not reflect the views of Microsoft Wed, 17 Apr 2019 03:24:23 GMT en-US hourly 1 Resuming posting https://docs.microsoft.com/archive/blogs/microsoftbob/resuming-posting Thu, 08 May 2014 23:30:12 GMT https://blogs.msdn.microsoft.com/microsoftbob/2014/05/08/resuming-posting/ Haven't posted for a while here and have been using https://bobleith.wordpress.com. I've decided to focus on my research work including simulation and financial markets on the bobleith.wordpress.com blog, but will get back to posting here on content related to .NET development and development architecture including TFS, Visual Studio.NET, and SQL development. I will be maintaining a copy of all posts here on the bobleith.wordpress.com blog as well.

I am putting together a series on agent based architecture for solving simulation problems that involves some .NET code including .NET CLR and hope to publish here in the next couple of weeks. Thank you for the interest.

]]> Now at 5.11 GB/s SQL backup to Null/3.12 GB/s local backup with I/O Accelerators https://docs.microsoft.com/archive/blogs/microsoftbob/now-at-5-11-gbs-sql-backup-to-null3-12-gbs-local-backup-with-io-accelerators Tue, 30 Oct 2012 18:21:04 GMT https://blogs.msdn.microsoft.com/microsoftbob/2012/10/30/now-at-5-11-gbs-sql-backup-to-null3-12-gbs-local-backup-with-io-accelerators/ This will be my last update for a while, but I did some more shuffling of the database files among the various PCIE SSD logical drives (HP I/O Accelerators which are HP-OEM Fusion-Io cards). See earlier post for info about the drives. In the latest test, I got up to 5.11 GB/s (5228.521 MB/sec) doing the SQL Backup to NUL device. Given a 40 GbE network card and adequate destinations, that same number should be achievable for a “real” network SQL backup. This is raw throughput. As pointed out earlier, factoring in the database compression efficiency of 40% and typical storage overhead, this is an effective or logical data rate closer to 9.0 GB/s.  Also, improved the prior raw 3.05 GB/s to 3.12 GB/s (3195.491 MB/Sec) to local drives including the ioDrives – SQL showed up to 8 GB/s combined read/write throughput spikes – this is significantly above the 6.4 QPI limit, so if any individual QPI bottlenecks are in play, they are not all-encompassing. This time the database was distributed across all of the Duos and for the local backup test it was backed up using the SQL Backup command to all of the Duos along with an additional non-Duo MLC Drive, and 4 SATA SSDs.

SQL command outputs included at end of post for skeptics … Smile. If still not convinced this is possible, check my YouTube video at the 7 minute mark with demo referenced by the 3.05 GB/s post.

This is utilizing 7 Duos spread across Windows-based Raid-0 and Raid-1 logical drives for the database (The ioDrives do not support hardware-based RAID).  In theory, RAID-1 is not be much slower than RAID-0 for read, so if all of the files were distributed evenly and read in parallel, the performance should be about twice that (7 * 1500 MB/s per Duo –>10.2 GB/s). Two issues seem to be complicating optimal performance:

1) The RAID-0 implementation from Windows Server – Not sure how it actually does this, but suspect that this involves buffering each data portion from the input drives together and then buffering out – normally this would be good for typical storage, but at these speeds almost any form of buffering just adds another bottleneck, particularly if it is saturating a QPI link. Based on that, better performance will probably be realized by distributing the files across non-Raid volumes, even the RAID-0. The database has dozens of files, but adding a lower level of granularity makes even file distribution more difficult. Depending on how low in the stack the RAID-0 is occurring, it could also be disadvantaged by the PCIE cards from different IOHs connected to different CPUs – can’t figure out which PCIE slots go to which IOHs – figure it is probably every other card or the first half of cards to one IOH and the other set to the other IOH.  Each logical read would need to bridge together the I/O from each device which would also involve a non-direct transfer from storage to CPU.

2) SQL Server backup internals – lack of knowledge is hurting me here. Not sure of all details of parallelization, but notice a balancing issue in that if one device has more data to read than another, it slows things down, but not to the degree as on the output balancing.  SQL also seems to perform the backup of the various database files in a manner not totally parallel – seemed to see some sequencing around portioned structures but not positive. It also does a final pass on the log, but that is not an issue here since there is noting in the transaction log at this point.

Trying to work through these issues will be time-consuming. Other things are more pressing, so will move onto other things and revisit later unless I stumble upon something in the interim. Thanks for the interest.

 

RESULTS (Local Backup)

/*------------------------
BACKUP DATABASE [tp_v5_dev] TO 
DISK = 'j:\sqlbackup\tp_v5_dev_2012_10_28_j1.bak',
DISK = 'k:\sqlbackup\tp_v5_dev_2012_10_28_k2.bak',
DISK=  'k:\sqlbackup\tp_v5_dev_2012_10_28_k3.bak',
DISK = 'k:\sqlbackup\tp_v5_dev_2012_10_28_k4.bak',
DISK = 'h:\sqlbackup\tp_v5_dev_2012_10_28_h5.bak',
DISK = 'h:\sqlbackup\tp_v5_dev_2012_10_28_h6.bak',
DISK = 'h:\sqlbackup\tp_v5_dev_2012_10_28_h6b.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_7.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_8.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_9.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_10.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_11.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_12.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_13.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_14.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_15.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_16.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_17.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_18.bak',
DISK = 'i:\sqlbackup\tp_v5_dev_2012_10_28_j1_b.bak',
DISK = 'k:\sqlbackup\tp_v5_dev_2012_10_28_k2_b.bak',
DISK=  'k:\sqlbackup\tp_v5_dev_2012_10_28_k3_b.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_k4_b.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_h5_b.bak',
DISK = 'h:\sqlbackup\tp_v5_dev_2012_10_28_h6_b.bak',
DISK = 'h:\sqlbackup\tp_v5_dev_2012_10_28_h6b_b.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_l_7_b.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_l_8_b.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_l_9_b.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_l_10_b.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_l_11_b.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_l_12_b.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_l_13_b.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_l_14_b.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_l_15_b.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_l_16_b.bak',
DISK = 'L:\sqlbackup\tp_v5_dev_2012_10_28_l_17_b.bak',
DISK = 'L:\sqlbackup\tp_v5_dev_2012_10_28_l_18_b.bak',
DISK=  'k:\sqlbackup\tp_v5_dev_2012_10_28_k3_c.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_k4_c.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_h5_c.bak',
DISK = 'h:\sqlbackup\tp_v5_dev_2012_10_28_h6b_c.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_7_c.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_8_c.bak',
DISK = 'L:\sqlbackup\tp_v5_dev_2012_10_28_l_9_c.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_10_c.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_11_c.bak',
DISK = 'j:\sqlbackup\tp_v5_dev_2012_10_28_l_12_c.bak',
DISK = 'g:\sqlbackup\tp_v5_dev_2012_10_28_l_13_c.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_14_c.bak',
DISK = 'h:\sqlbackup\tp_v5_dev_2012_10_28_l_15_c.bak',
DISK = 'k:\sqlbackup\tp_v5_dev_2012_10_28_l_16_c.bak',
DISK = 'f:\sqlbackup\tp_v5_dev_2012_10_28_l_17_c.bak',
DISK=  'J:\sqlbackup\tp_v5_dev_2012_10_28_k3_d.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_k4_d.bak',
DISK = 'F:\sqlbackup\tp_v5_dev_2012_10_28_l_8_d.bak',
DISK = 'l:\sqlbackup\tp_v5_dev_2012_10_28_l_10_d.bak',
DISK = 'H:\sqlbackup\tp_v5_dev_2012_10_28_l_11_d.bak',
DISK = 'H:\sqlbackup\tp_v5_dev_2012_10_28_l_12_d.bak',
DISK = 'K:\sqlbackup\tp_v5_dev_2012_10_28_l_14_d.bak'

-- 'NUL:' -- N'E:\SqlBackup\tp_v5_dev_20120708010002.BAK' WITH NOFORMAT, NOINIT, 
-- NAME = N'tp_v5_dev-Full Database Backup',
-- SKIP, NOREWIND, NOUNLOAD, 
--disk = 'NUL:'
with STATS = 10, NOINIT, SKIP, -- COPY_ONLY,
BUFFERCOUNT=1536, MAXTRANSFERSIZE=1048576, BLOCKSIZE=65536, NO_COMPRESSION
GO
------------------------*/
10 percent processed.
20 percent processed.
30 percent processed.
40 percent processed.
50 percent processed.
60 percent processed.
70 percent processed.
80 percent processed.
90 percent processed.
Processed 25288 pages for database 'tp_v5_dev', file 'TradingOptimizer_v2' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG01_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG02_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG03_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG04_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG05_PageFile01' on file 1.
Processed 17584 pages for database 'tp_v5_dev', file 'History09a' on file 1.
Processed 1680 pages for database 'tp_v5_dev', file 'History09b' on file 1.
Processed 24 pages for database 'tp_v5_dev', file 'History01_01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History01_02' on file 1.
Processed 24 pages for database 'tp_v5_dev', file 'History02a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History02b' on file 1.
Processed 24 pages for database 'tp_v5_dev', file 'History03a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History03b' on file 1.
Processed 2736 pages for database 'tp_v5_dev', file 'History04a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History04b' on file 1.
Processed 3400 pages for database 'tp_v5_dev', file 'History05a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History05b' on file 1.
Processed 5032 pages for database 'tp_v5_dev', file 'History06a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History06b' on file 1.
Processed 6936 pages for database 'tp_v5_dev', file 'History07a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History07b' on file 1.
Processed 10904 pages for database 'tp_v5_dev', file 'History08a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History08b' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG06_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG07_PageFile01' on file 1.
Processed 1814280 pages for database 'tp_v5_dev', file 'HistoryData01' on file 1.
Processed 1674784 pages for database 'tp_v5_dev', file 'HistoryData03' on file 1.
Processed 1657248 pages for database 'tp_v5_dev', file 'HistoryData05' on file 1.
Processed 1182680 pages for database 'tp_v5_dev', file 'LoadData02' on file 1.
Processed 1150464 pages for database 'tp_v5_dev', file 'LoadData04' on file 1.
Processed 1385080 pages for database 'tp_v5_dev', file 'LoadData01' on file 1.
Processed 1230520 pages for database 'tp_v5_dev', file 'LoadData03' on file 1.
Processed 357776 pages for database 'tp_v5_dev', file 'MiscData2' on file 1.
Processed 324864 pages for database 'tp_v5_dev', file 'MiscData1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG08_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG09_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG10_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG11_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG12_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG13_PageFile01' on file 1.
Processed 5112 pages for database 'tp_v5_dev', file 'History10_01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History10_02' on file 1.
Processed 56 pages for database 'tp_v5_dev', file 'Loadfg00a' on file 1.
Processed 56 pages for database 'tp_v5_dev', file 'Loadfg00b' on file 1.
Processed 32608 pages for database 'tp_v5_dev', file 'Loadfg01a' on file 1.
Processed 42000 pages for database 'tp_v5_dev', file 'Loadfg01b' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG14_PageFile01' on file 1.
Processed 5168 pages for database 'tp_v5_dev', file 'History09b1' on file 1.
Processed 464 pages for database 'tp_v5_dev', file 'History09b2' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'History09c1' on file 1.
Processed 5688 pages for database 'tp_v5_dev', file 'History09c2' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'History09d1' on file 1.
Processed 5760 pages for database 'tp_v5_dev', file 'History09d2' on file 1.
Processed 6488 pages for database 'tp_v5_dev', file 'History10a1' on file 1.
Processed 432 pages for database 'tp_v5_dev', file 'History10a2' on file 1.
Processed 8784 pages for database 'tp_v5_dev', file 'History10b1' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History10b2' on file 1.
Processed 20240 pages for database 'tp_v5_dev', file 'History10c2' on file 1.
Processed 55416 pages for database 'tp_v5_dev', file 'History10d2' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'Archive01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'Archive02' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'Archive03' on file 1.
Processed 31888 pages for database 'tp_v5_dev', file 'Loadfg02a' on file 1.
Processed 41488 pages for database 'tp_v5_dev', file 'Loadfg02b' on file 1.
Processed 43032 pages for database 'tp_v5_dev', file 'Loadfg03a' on file 1.
Processed 43448 pages for database 'tp_v5_dev', file 'Loadfg03b' on file 1.
Processed 40976 pages for database 'tp_v5_dev', file 'Loadfg04a' on file 1.
Processed 41248 pages for database 'tp_v5_dev', file 'Loadfg04b' on file 1.
Processed 34376 pages for database 'tp_v5_dev', file 'Loadfg05a' on file 1.
Processed 47120 pages for database 'tp_v5_dev', file 'Loadfg05b' on file 1.
Processed 44816 pages for database 'tp_v5_dev', file 'Loadfg06a' on file 1.
Processed 45072 pages for database 'tp_v5_dev', file 'Loadfg06b' on file 1.
Processed 44056 pages for database 'tp_v5_dev', file 'Loadfg07a' on file 1.
Processed 44344 pages for database 'tp_v5_dev', file 'Loadfg07b' on file 1.
Processed 41976 pages for database 'tp_v5_dev', file 'Loadfg08a' on file 1.
Processed 48648 pages for database 'tp_v5_dev', file 'Loadfg08b' on file 1.
Processed 50696 pages for database 'tp_v5_dev', file 'Loadfg09a' on file 1.
Processed 50696 pages for database 'tp_v5_dev', file 'Loadfg09b' on file 1.
Processed 51208 pages for database 'tp_v5_dev', file 'Loadfg10a' on file 1.
Processed 55432 pages for database 'tp_v5_dev', file 'Loadfg10b' on file 1.
Processed 56 pages for database 'tp_v5_dev', file 'Loadfg11d' on file 1.
Processed 46096 pages for database 'tp_v5_dev', file 'Loadfg11a' on file 1.
Processed 49672 pages for database 'tp_v5_dev', file 'Loadfg11b' on file 1.
Processed 46864 pages for database 'tp_v5_dev', file 'Loadfg12a' on file 1.
Processed 60424 pages for database 'tp_v5_dev', file 'Loadfg12b' on file 1.
Processed 41496 pages for database 'tp_v5_dev', file 'Loadfg13a' on file 1.
Processed 50808 pages for database 'tp_v5_dev', file 'Loadfg13b' on file 1.
Processed 35872 pages for database 'tp_v5_dev', file 'Loadfg14a' on file 1.
Processed 45192 pages for database 'tp_v5_dev', file 'Loadfg14b' on file 1.
Processed 45080 pages for database 'tp_v5_dev', file 'Loadfg15a' on file 1.
Processed 54352 pages for database 'tp_v5_dev', file 'Loadfg15b' on file 1.
Processed 39664 pages for database 'tp_v5_dev', file 'Loadfg16a' on file 1.
Processed 53256 pages for database 'tp_v5_dev', file 'Loadfg16b' on file 1.
Processed 39624 pages for database 'tp_v5_dev', file 'Loadfg17a' on file 1.
Processed 53256 pages for database 'tp_v5_dev', file 'Loadfg17b' on file 1.
Processed 41328 pages for database 'tp_v5_dev', file 'Loadfg18a' on file 1.
Processed 54784 pages for database 'tp_v5_dev', file 'Loadfg18b' on file 1.
Processed 116240 pages for database 'tp_v5_dev', file 'Loadfg19a' on file 1.
Processed 119688 pages for database 'tp_v5_dev', file 'Loadfg19b' on file 1.
Processed 131088 pages for database 'tp_v5_dev', file 'Loadfg20a' on file 1.
Processed 91656 pages for database 'tp_v5_dev', file 'Loadfg20b' on file 1.
Processed 71296 pages for database 'tp_v5_dev', file 'Loadfg21a' on file 1.
Processed 93704 pages for database 'tp_v5_dev', file 'Loadfg21b' on file 1.
Processed 74768 pages for database 'tp_v5_dev', file 'Loadfg22a' on file 1.
Processed 96832 pages for database 'tp_v5_dev', file 'Loadfg22b' on file 1.
Processed 71224 pages for database 'tp_v5_dev', file 'Loadfg23a' on file 1.
Processed 93704 pages for database 'tp_v5_dev', file 'Loadfg23b' on file 1.
Processed 76200 pages for database 'tp_v5_dev', file 'Loadfg24a' on file 1.
Processed 98312 pages for database 'tp_v5_dev', file 'Loadfg24b' on file 1.
Processed 64 pages for database 'tp_v5_dev', file 'SimIndex01' on file 1.
Processed 48 pages for database 'tp_v5_dev', file 'SimIndex02' on file 1.
Processed 48 pages for database 'tp_v5_dev', file 'SimIndex04' on file 1.
Processed 48 pages for database 'tp_v5_dev', file 'SimIndex05' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Sim42' on file 1.
Processed 65472 pages for database 'tp_v5_dev', file 'History11a1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History11a2' on file 1.
Processed 64 pages for database 'tp_v5_dev', file 'Intraday03' on file 1.
Processed 64 pages for database 'tp_v5_dev', file 'Intraday04' on file 1.
Processed 64744 pages for database 'tp_v5_dev', file 'History11b1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History11b2' on file 1.
Processed 73344 pages for database 'tp_v5_dev', file 'History11c2' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History11d1' on file 1.
Processed 34704 pages for database 'tp_v5_dev', file 'History11d2' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History12a1' on file 1.
Processed 144 pages for database 'tp_v5_dev', file 'History12a2' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History12b1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History12b2' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History12c1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History12c2' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History12d1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History12d2' on file 1.
Processed 94912 pages for database 'tp_v5_dev', file 'loadfg25a' on file 1.
Processed 105168 pages for database 'tp_v5_dev', file 'loadfg25b' on file 1.
Processed 112320 pages for database 'tp_v5_dev', file 'loadfg26a' on file 1.
Processed 114832 pages for database 'tp_v5_dev', file 'loadfg26b' on file 1.
Processed 129792 pages for database 'tp_v5_dev', file 'loadfg27a' on file 1.
Processed 145840 pages for database 'tp_v5_dev', file 'loadfg27b' on file 1.
Processed 114000 pages for database 'tp_v5_dev', file 'loadfg28a' on file 1.
Processed 115704 pages for database 'tp_v5_dev', file 'loadfg28b' on file 1.
Processed 114888 pages for database 'tp_v5_dev', file 'loadfg29a' on file 1.
Processed 116640 pages for database 'tp_v5_dev', file 'loadfg29b' on file 1.
Processed 127136 pages for database 'tp_v5_dev', file 'loadfg30a' on file 1.
Processed 137128 pages for database 'tp_v5_dev', file 'loadfg30b' on file 1.
Processed 114208 pages for database 'tp_v5_dev', file 'loadfg31a' on file 1.
Processed 114888 pages for database 'tp_v5_dev', file 'loadfg31b' on file 1.
Processed 130200 pages for database 'tp_v5_dev', file 'loadfg32a' on file 1.
Processed 139536 pages for database 'tp_v5_dev', file 'loadfg32b' on file 1.
Processed 114648 pages for database 'tp_v5_dev', file 'loadfg33a' on file 1.
Processed 126072 pages for database 'tp_v5_dev', file 'loadfg33b' on file 1.
Processed 113184 pages for database 'tp_v5_dev', file 'loadfg34a' on file 1.
Processed 122184 pages for database 'tp_v5_dev', file 'loadfg34b' on file 1.
Processed 105432 pages for database 'tp_v5_dev', file 'loadfg35a' on file 1.
Processed 126624 pages for database 'tp_v5_dev', file 'loadfg35b' on file 1.
Processed 111752 pages for database 'tp_v5_dev', file 'loadfg36a' on file 1.
Processed 121128 pages for database 'tp_v5_dev', file 'loadfg36b' on file 1.
Processed 150920 pages for database 'tp_v5_dev', file 'Loadfg37a' on file 1.
Processed 64000 pages for database 'tp_v5_dev', file 'Loadfg37b' on file 1.
Processed 113976 pages for database 'tp_v5_dev', file 'load38_01' on file 1.
Processed 102400 pages for database 'tp_v5_dev', file 'load38_02' on file 1.
Processed 121368 pages for database 'tp_v5_dev', file 'load39_01' on file 1.
Processed 115200 pages for database 'tp_v5_dev', file 'load39_02' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG15_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG16_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG17_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG18_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG19_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG20_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG21_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG22_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG23_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG24_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG25_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG26_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG27_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG28_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG29_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG30_PageFile01' on file 1.
Processed 44448 pages for database 'tp_v5_dev', file 'Interval_01' on file 1.
Processed 60544 pages for database 'tp_v5_dev', file 'Interval_fg01a' on file 1.
Processed 65752 pages for database 'tp_v5_dev', file 'Interval_fg01b' on file 1.
Processed 71600 pages for database 'tp_v5_dev', file 'Interval_fg01c' on file 1.
Processed 32168 pages for database 'tp_v5_dev', file 'Interval_02' on file 1.
Processed 58416 pages for database 'tp_v5_dev', file 'Interval_fg02a' on file 1.
Processed 58336 pages for database 'tp_v5_dev', file 'Interval_fg02b' on file 1.
Processed 75624 pages for database 'tp_v5_dev', file 'Interval_fg02c' on file 1.
Processed 35632 pages for database 'tp_v5_dev', file 'Interval_03' on file 1.
Processed 64488 pages for database 'tp_v5_dev', file 'Interval_fg03a' on file 1.
Processed 73336 pages for database 'tp_v5_dev', file 'Interval_fg03b' on file 1.
Processed 73776 pages for database 'tp_v5_dev', file 'Interval_fg03c' on file 1.
Processed 11456 pages for database 'tp_v5_dev', file 'Interval_04' on file 1.
Processed 30584 pages for database 'tp_v5_dev', file 'Interval_fg04a' on file 1.
Processed 30392 pages for database 'tp_v5_dev', file 'Interval_fg04b' on file 1.
Processed 31688 pages for database 'tp_v5_dev', file 'Interval_fg04c' on file 1.
Processed 38992 pages for database 'tp_v5_dev', file 'Interval_fg05a' on file 1.
Processed 38896 pages for database 'tp_v5_dev', file 'Interval_fg05b' on file 1.
Processed 39304 pages for database 'tp_v5_dev', file 'Interval_fg05c' on file 1.
Processed 22016 pages for database 'tp_v5_dev', file 'Interval_06' on file 1.
Processed 38704 pages for database 'tp_v5_dev', file 'Interval_fg06a' on file 1.
Processed 35408 pages for database 'tp_v5_dev', file 'Interval_fg06b' on file 1.
Processed 34840 pages for database 'tp_v5_dev', file 'Interval_fg06c' on file 1.
Processed 45920 pages for database 'tp_v5_dev', file 'Interval_fg07a' on file 1.
Processed 47080 pages for database 'tp_v5_dev', file 'Interval_fg07b' on file 1.
Processed 52416 pages for database 'tp_v5_dev', file 'Interval_fg07c' on file 1.
Processed 55120 pages for database 'tp_v5_dev', file 'Interval_fg08a' on file 1.
Processed 60064 pages for database 'tp_v5_dev', file 'Interval_fg08b' on file 1.
Processed 61808 pages for database 'tp_v5_dev', file 'Interval_fg08c' on file 1.
Processed 32392 pages for database 'tp_v5_dev', file 'Interval_09' on file 1.
Processed 64872 pages for database 'tp_v5_dev', file 'Interval_fg09a' on file 1.
Processed 63080 pages for database 'tp_v5_dev', file 'Interval_fg09b' on file 1.
Processed 60840 pages for database 'tp_v5_dev', file 'Interval_fg09c' on file 1.
Processed 13992 pages for database 'tp_v5_dev', file 'Interval_10' on file 1.
Processed 16272 pages for database 'tp_v5_dev', file 'Interval_fg10a' on file 1.
Processed 13352 pages for database 'tp_v5_dev', file 'Interval_fg10b' on file 1.
Processed 27160 pages for database 'tp_v5_dev', file 'Interval_fg10c' on file 1.
Processed 14720 pages for database 'tp_v5_dev', file 'Interval_11' on file 1.
Processed 16680 pages for database 'tp_v5_dev', file 'Interval_fg11a' on file 1.
Processed 28184 pages for database 'tp_v5_dev', file 'Interval_fg11b' on file 1.
Processed 13784 pages for database 'tp_v5_dev', file 'Interval_fg11c' on file 1.
Processed 15744 pages for database 'tp_v5_dev', file 'Interval_12' on file 1.
Processed 15040 pages for database 'tp_v5_dev', file 'Interval_fg12a' on file 1.
Processed 13936 pages for database 'tp_v5_dev', file 'Interval_fg12b' on file 1.
Processed 29944 pages for database 'tp_v5_dev', file 'Interval_fg12c' on file 1.
Processed 13200 pages for database 'tp_v5_dev', file 'Interval_13' on file 1.
Processed 23584 pages for database 'tp_v5_dev', file 'Interval_fg13a' on file 1.
Processed 12904 pages for database 'tp_v5_dev', file 'Interval_fg13b' on file 1.
Processed 26776 pages for database 'tp_v5_dev', file 'Interval_fg13c' on file 1.
Processed 15296 pages for database 'tp_v5_dev', file 'Interval_14' on file 1.
Processed 17840 pages for database 'tp_v5_dev', file 'Interval_fg14a' on file 1.
Processed 12320 pages for database 'tp_v5_dev', file 'Interval_fg14b' on file 1.
Processed 28320 pages for database 'tp_v5_dev', file 'Interval_fg14c' on file 1.
Processed 18400 pages for database 'tp_v5_dev', file 'Interval_fg15a' on file 1.
Processed 28928 pages for database 'tp_v5_dev', file 'Interval_fg15b' on file 1.
Processed 30744 pages for database 'tp_v5_dev', file 'Interval_fg15c' on file 1.
Processed 30088 pages for database 'tp_v5_dev', file 'Interval_fg16a' on file 1.
Processed 16184 pages for database 'tp_v5_dev', file 'Interval_fg16b' on file 1.
Processed 36504 pages for database 'tp_v5_dev', file 'Interval_fg16c' on file 1.
Processed 18968 pages for database 'tp_v5_dev', file 'Interval_17' on file 1.
Processed 14592 pages for database 'tp_v5_dev', file 'Interval_fg17a' on file 1.
Processed 30032 pages for database 'tp_v5_dev', file 'Interval_fg17b' on file 1.
Processed 31888 pages for database 'tp_v5_dev', file 'Interval_fg17c' on file 1.
Processed 22392 pages for database 'tp_v5_dev', file 'Interval_fg18a' on file 1.
Processed 30136 pages for database 'tp_v5_dev', file 'Interval_fg18b' on file 1.
Processed 37224 pages for database 'tp_v5_dev', file 'Interval_fg18c' on file 1.
Processed 33616 pages for database 'tp_v5_dev', file 'Interval_fg19a' on file 1.
Processed 30024 pages for database 'tp_v5_dev', file 'Interval_fg19b' on file 1.
Processed 29864 pages for database 'tp_v5_dev', file 'Interval_fg19c' on file 1.
Processed 36248 pages for database 'tp_v5_dev', file 'Interval_20a' on file 1.
Processed 27616 pages for database 'tp_v5_dev', file 'Interval_20b' on file 1.
Processed 27376 pages for database 'tp_v5_dev', file 'Interval_20c' on file 1.
Processed 12632 pages for database 'tp_v5_dev', file 'Interval_21' on file 1.
Processed 13080 pages for database 'tp_v5_dev', file 'Interval_fg21a' on file 1.
Processed 24608 pages for database 'tp_v5_dev', file 'Interval_fg21b' on file 1.
Processed 24288 pages for database 'tp_v5_dev', file 'Interval_fg21c' on file 1.
Processed 736 pages for database 'tp_v5_dev', file 'Interval_22b' on file 1.
Processed 992 pages for database 'tp_v5_dev', file 'Interval_fg22a' on file 1.
Processed 1032 pages for database 'tp_v5_dev', file 'Interval_fg22b' on file 1.
Processed 1088 pages for database 'tp_v5_dev', file 'Interval_fg22c' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg01_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg02_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg03_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg04_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg05_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg06_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg07_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg08_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg09_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg10_File01' on file 1.
Processed 768 pages for database 'tp_v5_dev', file 'Interval_fg23a' on file 1.
Processed 784 pages for database 'tp_v5_dev', file 'Interval_fg23b' on file 1.
Processed 784 pages for database 'tp_v5_dev', file 'Interval_fg23c' on file 1.
Processed 838952 pages for database 'tp_v5_dev', file 'Interval_fg24a' on file 1.
Processed 732784 pages for database 'tp_v5_dev', file 'Interval_fg24b' on file 1.
Processed 804256 pages for database 'tp_v5_dev', file 'Interval_fg24c' on file 1.
Processed 520 pages for database 'tp_v5_dev', file 'Interval_fg25a' on file 1.
Processed 520 pages for database 'tp_v5_dev', file 'Interval_fg25b' on file 1.
Processed 520 pages for database 'tp_v5_dev', file 'Interval_fg25c' on file 1.
Processed 219528 pages for database 'tp_v5_dev', file 'Load40_01' on file 1.
Processed 156968 pages for database 'tp_v5_dev', file 'Load41_01' on file 1.
Processed 169880 pages for database 'tp_v5_dev', file 'Load42_01' on file 1.
Processed 860640 pages for database 'tp_v5_dev', file 'Load43_01' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Load44_01' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Load45_01' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Load46_01' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Load47_01' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Load48_01' on file 1.
100 percent processed.
Processed 0 pages for database 'tp_v5_dev', file 'TradingOptimizer_v2_log' on file 1.
Processed 2 pages for database 'tp_v5_dev', file 'TradingOptimizerLog2' on file 1.
Processed 0 pages for database 'tp_v5_dev', file 'TradingOptimizerLog3' on file 1.
BACKUP DATABASE successfully processed 23884890 pages in 58.395 seconds (3195.491 MB/sec).

RESUTS (Backup TO NUL – forgot to include the query in the output)

10 percent processed.
20 percent processed.
30 percent processed.
40 percent processed.
50 percent processed.
60 percent processed.
70 percent processed.
80 percent processed.
90 percent processed.
Processed 25288 pages for database 'tp_v5_dev', file 'TradingOptimizer_v2' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG01_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG02_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG03_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG04_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG05_PageFile01' on file 1.
Processed 17584 pages for database 'tp_v5_dev', file 'History09a' on file 1.
Processed 1680 pages for database 'tp_v5_dev', file 'History09b' on file 1.
Processed 24 pages for database 'tp_v5_dev', file 'History01_01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History01_02' on file 1.
Processed 24 pages for database 'tp_v5_dev', file 'History02a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History02b' on file 1.
Processed 24 pages for database 'tp_v5_dev', file 'History03a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History03b' on file 1.
Processed 2736 pages for database 'tp_v5_dev', file 'History04a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History04b' on file 1.
Processed 3400 pages for database 'tp_v5_dev', file 'History05a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History05b' on file 1.
Processed 5032 pages for database 'tp_v5_dev', file 'History06a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History06b' on file 1.
Processed 6936 pages for database 'tp_v5_dev', file 'History07a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History07b' on file 1.
Processed 10904 pages for database 'tp_v5_dev', file 'History08a' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History08b' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG06_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG07_PageFile01' on file 1.
Processed 1814280 pages for database 'tp_v5_dev', file 'HistoryData01' on file 1.
Processed 1674784 pages for database 'tp_v5_dev', file 'HistoryData03' on file 1.
Processed 1657248 pages for database 'tp_v5_dev', file 'HistoryData05' on file 1.
Processed 1182680 pages for database 'tp_v5_dev', file 'LoadData02' on file 1.
Processed 1150464 pages for database 'tp_v5_dev', file 'LoadData04' on file 1.
Processed 1385080 pages for database 'tp_v5_dev', file 'LoadData01' on file 1.
Processed 1230520 pages for database 'tp_v5_dev', file 'LoadData03' on file 1.
Processed 357776 pages for database 'tp_v5_dev', file 'MiscData2' on file 1.
Processed 324864 pages for database 'tp_v5_dev', file 'MiscData1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG08_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG09_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG10_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG11_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG12_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG13_PageFile01' on file 1.
Processed 5112 pages for database 'tp_v5_dev', file 'History10_01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History10_02' on file 1.
Processed 56 pages for database 'tp_v5_dev', file 'Loadfg00a' on file 1.
Processed 56 pages for database 'tp_v5_dev', file 'Loadfg00b' on file 1.
Processed 32608 pages for database 'tp_v5_dev', file 'Loadfg01a' on file 1.
Processed 42000 pages for database 'tp_v5_dev', file 'Loadfg01b' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG14_PageFile01' on file 1.
Processed 5168 pages for database 'tp_v5_dev', file 'History09b1' on file 1.
Processed 464 pages for database 'tp_v5_dev', file 'History09b2' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'History09c1' on file 1.
Processed 5688 pages for database 'tp_v5_dev', file 'History09c2' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'History09d1' on file 1.
Processed 5760 pages for database 'tp_v5_dev', file 'History09d2' on file 1.
Processed 6488 pages for database 'tp_v5_dev', file 'History10a1' on file 1.
Processed 432 pages for database 'tp_v5_dev', file 'History10a2' on file 1.
Processed 8784 pages for database 'tp_v5_dev', file 'History10b1' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History10b2' on file 1.
Processed 20240 pages for database 'tp_v5_dev', file 'History10c2' on file 1.
Processed 55416 pages for database 'tp_v5_dev', file 'History10d2' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'Archive01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'Archive02' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'Archive03' on file 1.
Processed 31888 pages for database 'tp_v5_dev', file 'Loadfg02a' on file 1.
Processed 41488 pages for database 'tp_v5_dev', file 'Loadfg02b' on file 1.
Processed 43032 pages for database 'tp_v5_dev', file 'Loadfg03a' on file 1.
Processed 43448 pages for database 'tp_v5_dev', file 'Loadfg03b' on file 1.
Processed 40976 pages for database 'tp_v5_dev', file 'Loadfg04a' on file 1.
Processed 41248 pages for database 'tp_v5_dev', file 'Loadfg04b' on file 1.
Processed 34376 pages for database 'tp_v5_dev', file 'Loadfg05a' on file 1.
Processed 47120 pages for database 'tp_v5_dev', file 'Loadfg05b' on file 1.
Processed 44816 pages for database 'tp_v5_dev', file 'Loadfg06a' on file 1.
Processed 45072 pages for database 'tp_v5_dev', file 'Loadfg06b' on file 1.
Processed 44056 pages for database 'tp_v5_dev', file 'Loadfg07a' on file 1.
Processed 44344 pages for database 'tp_v5_dev', file 'Loadfg07b' on file 1.
Processed 41976 pages for database 'tp_v5_dev', file 'Loadfg08a' on file 1.
Processed 48648 pages for database 'tp_v5_dev', file 'Loadfg08b' on file 1.
Processed 50696 pages for database 'tp_v5_dev', file 'Loadfg09a' on file 1.
Processed 50696 pages for database 'tp_v5_dev', file 'Loadfg09b' on file 1.
Processed 51208 pages for database 'tp_v5_dev', file 'Loadfg10a' on file 1.
Processed 55432 pages for database 'tp_v5_dev', file 'Loadfg10b' on file 1.
Processed 56 pages for database 'tp_v5_dev', file 'Loadfg11d' on file 1.
Processed 46096 pages for database 'tp_v5_dev', file 'Loadfg11a' on file 1.
Processed 49672 pages for database 'tp_v5_dev', file 'Loadfg11b' on file 1.
Processed 46864 pages for database 'tp_v5_dev', file 'Loadfg12a' on file 1.
Processed 60424 pages for database 'tp_v5_dev', file 'Loadfg12b' on file 1.
Processed 41496 pages for database 'tp_v5_dev', file 'Loadfg13a' on file 1.
Processed 50808 pages for database 'tp_v5_dev', file 'Loadfg13b' on file 1.
Processed 35872 pages for database 'tp_v5_dev', file 'Loadfg14a' on file 1.
Processed 45192 pages for database 'tp_v5_dev', file 'Loadfg14b' on file 1.
Processed 45080 pages for database 'tp_v5_dev', file 'Loadfg15a' on file 1.
Processed 54352 pages for database 'tp_v5_dev', file 'Loadfg15b' on file 1.
Processed 39664 pages for database 'tp_v5_dev', file 'Loadfg16a' on file 1.
Processed 53256 pages for database 'tp_v5_dev', file 'Loadfg16b' on file 1.
Processed 39624 pages for database 'tp_v5_dev', file 'Loadfg17a' on file 1.
Processed 53256 pages for database 'tp_v5_dev', file 'Loadfg17b' on file 1.
Processed 41328 pages for database 'tp_v5_dev', file 'Loadfg18a' on file 1.
Processed 54784 pages for database 'tp_v5_dev', file 'Loadfg18b' on file 1.
Processed 116240 pages for database 'tp_v5_dev', file 'Loadfg19a' on file 1.
Processed 119688 pages for database 'tp_v5_dev', file 'Loadfg19b' on file 1.
Processed 131088 pages for database 'tp_v5_dev', file 'Loadfg20a' on file 1.
Processed 91656 pages for database 'tp_v5_dev', file 'Loadfg20b' on file 1.
Processed 71296 pages for database 'tp_v5_dev', file 'Loadfg21a' on file 1.
Processed 93704 pages for database 'tp_v5_dev', file 'Loadfg21b' on file 1.
Processed 74768 pages for database 'tp_v5_dev', file 'Loadfg22a' on file 1.
Processed 96832 pages for database 'tp_v5_dev', file 'Loadfg22b' on file 1.
Processed 71224 pages for database 'tp_v5_dev', file 'Loadfg23a' on file 1.
Processed 93704 pages for database 'tp_v5_dev', file 'Loadfg23b' on file 1.
Processed 76200 pages for database 'tp_v5_dev', file 'Loadfg24a' on file 1.
Processed 98312 pages for database 'tp_v5_dev', file 'Loadfg24b' on file 1.
Processed 64 pages for database 'tp_v5_dev', file 'SimIndex01' on file 1.
Processed 48 pages for database 'tp_v5_dev', file 'SimIndex02' on file 1.
Processed 48 pages for database 'tp_v5_dev', file 'SimIndex04' on file 1.
Processed 48 pages for database 'tp_v5_dev', file 'SimIndex05' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Sim42' on file 1.
Processed 65472 pages for database 'tp_v5_dev', file 'History11a1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History11a2' on file 1.
Processed 64 pages for database 'tp_v5_dev', file 'Intraday03' on file 1.
Processed 64 pages for database 'tp_v5_dev', file 'Intraday04' on file 1.
Processed 64744 pages for database 'tp_v5_dev', file 'History11b1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History11b2' on file 1.
Processed 73344 pages for database 'tp_v5_dev', file 'History11c2' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History11d1' on file 1.
Processed 34704 pages for database 'tp_v5_dev', file 'History11d2' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History12a1' on file 1.
Processed 144 pages for database 'tp_v5_dev', file 'History12a2' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History12b1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History12b2' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History12c1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History12c2' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'History12d1' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'History12d2' on file 1.
Processed 94912 pages for database 'tp_v5_dev', file 'loadfg25a' on file 1.
Processed 105168 pages for database 'tp_v5_dev', file 'loadfg25b' on file 1.
Processed 112320 pages for database 'tp_v5_dev', file 'loadfg26a' on file 1.
Processed 114832 pages for database 'tp_v5_dev', file 'loadfg26b' on file 1.
Processed 129792 pages for database 'tp_v5_dev', file 'loadfg27a' on file 1.
Processed 145840 pages for database 'tp_v5_dev', file 'loadfg27b' on file 1.
Processed 114000 pages for database 'tp_v5_dev', file 'loadfg28a' on file 1.
Processed 115704 pages for database 'tp_v5_dev', file 'loadfg28b' on file 1.
Processed 114888 pages for database 'tp_v5_dev', file 'loadfg29a' on file 1.
Processed 116640 pages for database 'tp_v5_dev', file 'loadfg29b' on file 1.
Processed 127136 pages for database 'tp_v5_dev', file 'loadfg30a' on file 1.
Processed 137128 pages for database 'tp_v5_dev', file 'loadfg30b' on file 1.
Processed 114208 pages for database 'tp_v5_dev', file 'loadfg31a' on file 1.
Processed 114888 pages for database 'tp_v5_dev', file 'loadfg31b' on file 1.
Processed 130200 pages for database 'tp_v5_dev', file 'loadfg32a' on file 1.
Processed 139536 pages for database 'tp_v5_dev', file 'loadfg32b' on file 1.
Processed 114648 pages for database 'tp_v5_dev', file 'loadfg33a' on file 1.
Processed 126072 pages for database 'tp_v5_dev', file 'loadfg33b' on file 1.
Processed 113184 pages for database 'tp_v5_dev', file 'loadfg34a' on file 1.
Processed 122184 pages for database 'tp_v5_dev', file 'loadfg34b' on file 1.
Processed 105432 pages for database 'tp_v5_dev', file 'loadfg35a' on file 1.
Processed 126624 pages for database 'tp_v5_dev', file 'loadfg35b' on file 1.
Processed 111752 pages for database 'tp_v5_dev', file 'loadfg36a' on file 1.
Processed 121128 pages for database 'tp_v5_dev', file 'loadfg36b' on file 1.
Processed 150920 pages for database 'tp_v5_dev', file 'Loadfg37a' on file 1.
Processed 64000 pages for database 'tp_v5_dev', file 'Loadfg37b' on file 1.
Processed 113976 pages for database 'tp_v5_dev', file 'load38_01' on file 1.
Processed 102400 pages for database 'tp_v5_dev', file 'load38_02' on file 1.
Processed 121368 pages for database 'tp_v5_dev', file 'load39_01' on file 1.
Processed 115200 pages for database 'tp_v5_dev', file 'load39_02' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG15_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG16_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG17_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG18_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG19_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG20_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG21_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG22_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG23_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG24_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG25_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG26_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG27_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG28_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG29_PageFile01' on file 1.
Processed 32 pages for database 'tp_v5_dev', file 'PageField_FG30_PageFile01' on file 1.
Processed 44448 pages for database 'tp_v5_dev', file 'Interval_01' on file 1.
Processed 60544 pages for database 'tp_v5_dev', file 'Interval_fg01a' on file 1.
Processed 65752 pages for database 'tp_v5_dev', file 'Interval_fg01b' on file 1.
Processed 71600 pages for database 'tp_v5_dev', file 'Interval_fg01c' on file 1.
Processed 32168 pages for database 'tp_v5_dev', file 'Interval_02' on file 1.
Processed 58416 pages for database 'tp_v5_dev', file 'Interval_fg02a' on file 1.
Processed 58336 pages for database 'tp_v5_dev', file 'Interval_fg02b' on file 1.
Processed 75624 pages for database 'tp_v5_dev', file 'Interval_fg02c' on file 1.
Processed 35632 pages for database 'tp_v5_dev', file 'Interval_03' on file 1.
Processed 64488 pages for database 'tp_v5_dev', file 'Interval_fg03a' on file 1.
Processed 73336 pages for database 'tp_v5_dev', file 'Interval_fg03b' on file 1.
Processed 73776 pages for database 'tp_v5_dev', file 'Interval_fg03c' on file 1.
Processed 11456 pages for database 'tp_v5_dev', file 'Interval_04' on file 1.
Processed 30584 pages for database 'tp_v5_dev', file 'Interval_fg04a' on file 1.
Processed 30392 pages for database 'tp_v5_dev', file 'Interval_fg04b' on file 1.
Processed 31688 pages for database 'tp_v5_dev', file 'Interval_fg04c' on file 1.
Processed 38992 pages for database 'tp_v5_dev', file 'Interval_fg05a' on file 1.
Processed 38896 pages for database 'tp_v5_dev', file 'Interval_fg05b' on file 1.
Processed 39304 pages for database 'tp_v5_dev', file 'Interval_fg05c' on file 1.
Processed 22016 pages for database 'tp_v5_dev', file 'Interval_06' on file 1.
Processed 38704 pages for database 'tp_v5_dev', file 'Interval_fg06a' on file 1.
Processed 35408 pages for database 'tp_v5_dev', file 'Interval_fg06b' on file 1.
Processed 34840 pages for database 'tp_v5_dev', file 'Interval_fg06c' on file 1.
Processed 45920 pages for database 'tp_v5_dev', file 'Interval_fg07a' on file 1.
Processed 47080 pages for database 'tp_v5_dev', file 'Interval_fg07b' on file 1.
Processed 52416 pages for database 'tp_v5_dev', file 'Interval_fg07c' on file 1.
Processed 55120 pages for database 'tp_v5_dev', file 'Interval_fg08a' on file 1.
Processed 60064 pages for database 'tp_v5_dev', file 'Interval_fg08b' on file 1.
Processed 61808 pages for database 'tp_v5_dev', file 'Interval_fg08c' on file 1.
Processed 32392 pages for database 'tp_v5_dev', file 'Interval_09' on file 1.
Processed 64872 pages for database 'tp_v5_dev', file 'Interval_fg09a' on file 1.
Processed 63080 pages for database 'tp_v5_dev', file 'Interval_fg09b' on file 1.
Processed 60840 pages for database 'tp_v5_dev', file 'Interval_fg09c' on file 1.
Processed 13992 pages for database 'tp_v5_dev', file 'Interval_10' on file 1.
Processed 16272 pages for database 'tp_v5_dev', file 'Interval_fg10a' on file 1.
Processed 13352 pages for database 'tp_v5_dev', file 'Interval_fg10b' on file 1.
Processed 27160 pages for database 'tp_v5_dev', file 'Interval_fg10c' on file 1.
Processed 14720 pages for database 'tp_v5_dev', file 'Interval_11' on file 1.
Processed 16680 pages for database 'tp_v5_dev', file 'Interval_fg11a' on file 1.
Processed 28184 pages for database 'tp_v5_dev', file 'Interval_fg11b' on file 1.
Processed 13784 pages for database 'tp_v5_dev', file 'Interval_fg11c' on file 1.
Processed 15744 pages for database 'tp_v5_dev', file 'Interval_12' on file 1.
Processed 15040 pages for database 'tp_v5_dev', file 'Interval_fg12a' on file 1.
Processed 13936 pages for database 'tp_v5_dev', file 'Interval_fg12b' on file 1.
Processed 29944 pages for database 'tp_v5_dev', file 'Interval_fg12c' on file 1.
Processed 13200 pages for database 'tp_v5_dev', file 'Interval_13' on file 1.
Processed 23584 pages for database 'tp_v5_dev', file 'Interval_fg13a' on file 1.
Processed 12904 pages for database 'tp_v5_dev', file 'Interval_fg13b' on file 1.
Processed 26776 pages for database 'tp_v5_dev', file 'Interval_fg13c' on file 1.
Processed 15296 pages for database 'tp_v5_dev', file 'Interval_14' on file 1.
Processed 17840 pages for database 'tp_v5_dev', file 'Interval_fg14a' on file 1.
Processed 12320 pages for database 'tp_v5_dev', file 'Interval_fg14b' on file 1.
Processed 28320 pages for database 'tp_v5_dev', file 'Interval_fg14c' on file 1.
Processed 18400 pages for database 'tp_v5_dev', file 'Interval_fg15a' on file 1.
Processed 28928 pages for database 'tp_v5_dev', file 'Interval_fg15b' on file 1.
Processed 30744 pages for database 'tp_v5_dev', file 'Interval_fg15c' on file 1.
Processed 30088 pages for database 'tp_v5_dev', file 'Interval_fg16a' on file 1.
Processed 16184 pages for database 'tp_v5_dev', file 'Interval_fg16b' on file 1.
Processed 36504 pages for database 'tp_v5_dev', file 'Interval_fg16c' on file 1.
Processed 18968 pages for database 'tp_v5_dev', file 'Interval_17' on file 1.
Processed 14592 pages for database 'tp_v5_dev', file 'Interval_fg17a' on file 1.
Processed 30032 pages for database 'tp_v5_dev', file 'Interval_fg17b' on file 1.
Processed 31888 pages for database 'tp_v5_dev', file 'Interval_fg17c' on file 1.
Processed 22392 pages for database 'tp_v5_dev', file 'Interval_fg18a' on file 1.
Processed 30136 pages for database 'tp_v5_dev', file 'Interval_fg18b' on file 1.
Processed 37224 pages for database 'tp_v5_dev', file 'Interval_fg18c' on file 1.
Processed 33616 pages for database 'tp_v5_dev', file 'Interval_fg19a' on file 1.
Processed 30024 pages for database 'tp_v5_dev', file 'Interval_fg19b' on file 1.
Processed 29864 pages for database 'tp_v5_dev', file 'Interval_fg19c' on file 1.
Processed 36248 pages for database 'tp_v5_dev', file 'Interval_20a' on file 1.
Processed 27616 pages for database 'tp_v5_dev', file 'Interval_20b' on file 1.
Processed 27376 pages for database 'tp_v5_dev', file 'Interval_20c' on file 1.
Processed 12632 pages for database 'tp_v5_dev', file 'Interval_21' on file 1.
Processed 13080 pages for database 'tp_v5_dev', file 'Interval_fg21a' on file 1.
Processed 24608 pages for database 'tp_v5_dev', file 'Interval_fg21b' on file 1.
Processed 24288 pages for database 'tp_v5_dev', file 'Interval_fg21c' on file 1.
Processed 736 pages for database 'tp_v5_dev', file 'Interval_22b' on file 1.
Processed 992 pages for database 'tp_v5_dev', file 'Interval_fg22a' on file 1.
Processed 1032 pages for database 'tp_v5_dev', file 'Interval_fg22b' on file 1.
Processed 1088 pages for database 'tp_v5_dev', file 'Interval_fg22c' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg01_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg02_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg03_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg04_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg05_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg06_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg07_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg08_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg09_File01' on file 1.
Processed 40 pages for database 'tp_v5_dev', file 'MatchType_Fg10_File01' on file 1.
Processed 768 pages for database 'tp_v5_dev', file 'Interval_fg23a' on file 1.
Processed 784 pages for database 'tp_v5_dev', file 'Interval_fg23b' on file 1.
Processed 784 pages for database 'tp_v5_dev', file 'Interval_fg23c' on file 1.
Processed 838952 pages for database 'tp_v5_dev', file 'Interval_fg24a' on file 1.
Processed 732784 pages for database 'tp_v5_dev', file 'Interval_fg24b' on file 1.
Processed 804256 pages for database 'tp_v5_dev', file 'Interval_fg24c' on file 1.
Processed 520 pages for database 'tp_v5_dev', file 'Interval_fg25a' on file 1.
Processed 520 pages for database 'tp_v5_dev', file 'Interval_fg25b' on file 1.
Processed 520 pages for database 'tp_v5_dev', file 'Interval_fg25c' on file 1.
Processed 219528 pages for database 'tp_v5_dev', file 'Load40_01' on file 1.
Processed 156968 pages for database 'tp_v5_dev', file 'Load41_01' on file 1.
Processed 169880 pages for database 'tp_v5_dev', file 'Load42_01' on file 1.
Processed 860640 pages for database 'tp_v5_dev', file 'Load43_01' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Load44_01' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Load45_01' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Load46_01' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Load47_01' on file 1.
Processed 16 pages for database 'tp_v5_dev', file 'Load48_01' on file 1.
100 percent processed.
Processed 0 pages for database 'tp_v5_dev', file 'TradingOptimizer_v2_log' on file 1.
Processed 2 pages for database 'tp_v5_dev', file 'TradingOptimizerLog2' on file 1.
Processed 0 pages for database 'tp_v5_dev', file 'TradingOptimizerLog3' on file 1.
BACKUP DATABASE successfully processed 23884890 pages in 35.689 seconds (5228.521 MB/sec).

]]> More clarifications on 3.05 GB/s SQL backup potential for much higher throughput https://docs.microsoft.com/archive/blogs/microsoftbob/more-clarifications-on-3-05-gbs-sql-backup-potential-for-much-higher-throughput Mon, 29 Oct 2012 16:00:49 GMT https://blogs.msdn.microsoft.com/microsoftbob/2012/10/29/more-clarifications-on-3-05-gbs-sql-backup-potential-for-much-higher-throughput/ A couple of clarifications - including potential for much greater backup throughput beyond 3 GB/s up to 9 GB/s for this hardware:

1) Contrary to the video, 10 GbE is not the fastest practical Ethernet for server-to-server communications. 40 GbE network cards and switches are available and affordable. It is feasible to achieve 5GB/s using a PCIE Gen 2 40 GbE card as well as via a 4 port 10 GbE adapter configured suitably. Mellanox produces a Gen 2 PCIE card supporting 40 Gb/s Ethernet and 32 Gb/s (actual) QDR InfiniBand as well as a Gen 3 version with FDR InfiniBand throughput of 56 Gb/s (54.54 Gb/s actual) providing close to 7 GB/s bandwidth. See https://www.mellanox.com/content/pages.php?pg=infiniband_cards_overview&menu_section=41. 

However, Just three x8 Duo2s which can be hosted in many servers as small as 2U generates up to 9 GB/s throughput saturating 56 Gb/s InfiniBand, the fastest network interface card having practical availability. PCIE SSD adoption means the network is now the bottleneck rather than storage I/O in the area of server-to-server data transfer.

2) Although backup compression is not used in the testing due to CPU limitations, database row/page compression is used achieving about 40% space reduction. This makes “real-world” effective backup throughput closer to 5 GB/s. Extrapolating that metric pushes logical throughput into the 15 GB/s range for the tested configuration if the current bottleneck (next item) is resolved.

3) CPUs are averaging under 50% utilization in my testing so that does not seem to be the bottleneck. I suspect a QPI (Quick Path Interconnect) configuration issue. See https://www.qdpma.com/systemarchitecture/systemarchitecture_qpi.html. The aggregate total bandwidth in a 2-way Nehalem architecture is 25.6 GB/s accommodated through 4 x 6.4 GB/s QPI links. The QPI links comprise:

a. One between the two I/O hubs (IOH) (each of which connects directly to half of the PCIE cards)

b. One each (total of two) between each IOH and a processor socket (i.e. NUMA node),

c. One between the sockets themselves.

As I understand, If data transfer is monopolizing the IOH-to-IOH or CPU-to-CPU links rather than affinitized between each processor and the processor’s directly-connected IOH this limits throughput to the individual QPI speed of 6.4 GB/s. If PCIE/NUMA configuration causing QPI link saturation is the bottleneck, then resolving should enable a SQL backup of a database stored on 6 Duos (1.5 GB/s read rate) to achieve 9 GB/s distributed among collectively-adequate destination targets using a pair of 40 GbE network cards. Using the SQL Backup to NULL operation can help verify. I realize there are other possible culprits – i.e. inefficient cache use, SQL Server specific NUMA configuration - Open to suggestions. The article https://www.micron.com/~/media/Documents/Products/White%20Paper/pcie_ssd_performance_hi.pdf provides recommendations for optimizing PCIE configuration for throughput.

Note: Due to an unexpected health crisis in our family, I need to re-prioritize and will be slow for next several days reviewing responses to the blog and associated video.

Thank you.

]]> A couple of corrections on video https://docs.microsoft.com/archive/blogs/microsoftbob/a-couple-of-corrections-on-video Mon, 29 Oct 2012 01:32:34 GMT https://blogs.msdn.microsoft.com/microsoftbob/2012/10/29/a-couple-of-corrections-on-video/
  • I should have said that 40 Gbe and 100 Gbe are not widely implemented rather than not available  – they are available. But, even 100 GbE is only adequate for 12 GBPS – and with Gen 3 PCIE SSD reading at 3.0 GBps +, that will easily be overrun by newer servers that can utilize 5 or more of those cards. Ultimately the solution to the network bottleneck issue probably involves utilizing enough parallel multiple network interface cards/links together to match the PCIE speeds or extending PCIE as a network transfer itself. See https://www.networkworld.com/news/tech/2012/030112-pci-express-256848.html
  • The effective transfer rate is actually better than the 4GBps proposed when factoring in the effectiveness of database page/row compression. As pointed out in the blog entry, my sample database is storing closer to 1/2 TB of logical data when considering the compression ratios achieved. SQL Backup reads compressed data natively and backs it up as is without decompressing, thus increasing effective read throughput by the effectiveness of the database compression with no adverse impact on the backup performance. It is important to understand the difference between backup compression and database compression – backup compression does occur while the database is being backed up and reduces the amount of storage needed for the target destination at the cost of CPU – however data in a database already compressed using page/row database compression is simply backed up natively.
  • The title should probably end with “using HP IO Accelerators” rather than Fusion-IO cards since I am using the HP-branded cards, but my feeling is that those new to the technology may not realize that the HP cards are actually just OEM versions of Fusion-IO cards.

    • There are probably a few other misstatements, feel free to let me know any others.

    Thanks.

    ]]>         1/4 TB SQL Database backup in under 1 Minute!!! (3.05 GBPS transfer rate) https://docs.microsoft.com/archive/blogs/microsoftbob/14-tb-sql-database-backup-in-under-1-minute-3-05-gbps-transfer-rate Sun, 28 Oct 2012 23:57:17 GMT https://blogs.msdn.microsoft.com/microsoftbob/2012/10/28/14-tb-sql-database-backup-in-under-1-minute-3-05-gbps-transfer-rate/ With the addition of another SATA SSD as well as another SLC Duo to serve as the target destination, the same SQL database which now has a total of 233 GB of used space was backed up in less than 1 minute using the same server referenced in my prior 2.0 GBPS backup post (https://blogs.msdn.com/b/microsoftbob/archive/2012/10/18/2gbps-backup-on-12-core-server-with-7-fusion-io-cards.aspx).

    SQL Query Analyzer showed the throughput rate as 3119.9224 MBPS which rounds out to 3.05 GBPS, about 2.5 times the capacity of 10Gb Ethernet and fast enough to utilize 75% of 32gb InfiniBand often used in enterprise cloud infrastructures.

    sql-snip1

     

    sql-snip1

    Note that the effective throughput was actually closer to 3 GBPS on the prior test rather than 2 GBPS and for the same rationale the effective rate of this test was close to 4.0 GBPS. This is based on the fact that there will be about 20 – 30% pages in a typical database – with or without compression – that is encompassed by a data structure (not marked as available) but not having to be backed up.

    sql-snip0

     

    I think (but am not yet sure) that the backup stats are only counting pages that are backed up and some pages in the database, although shown as is use, may not actually contain any data to backup. Whether you go with the higher or lower figure, the transfer rate is still remarkable.  Even using the lower 3.0 GBPS actual transfer rate, this works out to 180GB Per Minute and nearly11TB per hour.  Not bad considering that an Oracle Exabyte Database Machine (https://www.oracle.com/technetwork/database/features/availability/maa-tech-wp-sundbm-backup-11202-183503.pdf) only achieves 17 TB per hour on InfiniBand with hardware costing several million dollars, whereas my total investment here is under 50K. Even considering retail pricing on the cards, the total cost of the server is less than 180K. If we trust the backup to NULL would provide the maximum throughput of 5GBPS over 40GbPS InfiniBand , this would beat the Oracle configuration at 18TB Per Hour.  In terms of TPC-type terminology, we could say this server is providing each 1TB per hour backup speed at a cost of 10K.

    This was produced by a modest 2 way 2.8 GHz X5660 configuration with 12 cores, 108GB, 8 HP IO Accelerator Cards (HP Branded Fusion-IO cards), 4 SATA SSDS (Intel and Samsung) with the O/S on 3 SAS mechanical drives using Raid-5. Details of the IO Accelerator configuration is in the prior post with the exception that the non-functioning SLC DUO is functioning in this test and the addition of 2 SATA SSDs. The additional devices were used to supplement the target backup destination which was the main limiting factor on the prior benchmark.

    Backup to NUL device registered at nearly 4 GBPS, implying the bottleneck on the receiving end still even though the theoretical bandwidth of the destination should be over 4 GBPS. Based on testing using the NUL device as a backup target, It seems that this server is not able to provide more than around 8GBPS throughput combined read/write – even though the devices collectively added together are capable of over 10 GBPS – perhaps this is a BUS limitation somewhere between the PCIE bus and the processors.

    The implications of this are far reaching:

    Using an 8-way server with 128 or more cores would most likely allow the CPUs to compress fast enough for the data and double this to 36TB per hour, since such backup compression occurs before the data is sent over the wire. My database is actually achieving about 40% space reduction due to page and row compression and has multiple tables with over 1 billion rows, so in reality it is 1/2 TB of data. With a 128+ core system, if we factor in typical gains from backup and database compression, dedicated all the Accelerator cards to the database with an InfiniBand card using PCIE GEN 3 over a 56 Gb InfiniBand link, we could expect over 100TB per hour for a typical SQL Server backup – from a single 8-way 8U Server – costing less than 300K. This is using the first generation cards, the second generation cards double this.  Such an 8U server with PCIE GEN 3 cards could potentially saturate a 120 GB InfiniBand link with a 240TB per hour, backing up 1PB in a little over 4 hours.

    To get an idea of how much 1PB is, see this link - https://gizmodo.com/5309889/how-large-is-a-petabyte – it is just 1/20th of the amount of data processed by Google on a single day and 2/3rds of all the photos on Facebook.

    For more background on the configuration and methodology, see my earlier post at https://blogs.msdn.com/b/microsoftbob/archive/2012/10/18/2gbps-backup-on-12-core-server-with-7-fusion-io-cards.aspx.  You can see a demonstration of the backup and discussion. If you want to just see the actual backup execution, skip to the 7 minute mark. This is my first attempt at a youtube video, so please be merciful -

     

    ]]>         Quick update on IO Accelerator (testing) https://docs.microsoft.com/archive/blogs/microsoftbob/quick-update-on-io-accelerator-testing Fri, 19 Oct 2012 18:50:00 GMT https://blogs.msdn.microsoft.com/microsoftbob/2012/10/19/quick-update-on-io-accelerator-testing/ Follow-up from last post on 2.0 GBPS backup throughput - was able to update the firmware painlessly for the non-functional card from a Win 7 X64 machine so that it can now be updated on the HP Server to the latest HP-branded IO Accelerator driver. To my pleasant surprise, the last generic driver version (2.3.10) supported firmware updates all the way back from 2.1.0, the version associated with the firmware on the Card. As expected, the IO Accelerator card was virtually unused. It shows 0 GB Physical write. This 320GB HP IO Accelerator SLC Duo card was obtained for 1,500.00 from a liquidator on EBAY in September, the least I have ever paid for one of these. The price for a brand new HP SLC Duo on HP site? $18,359.00 so this was a savings of $16,859.00 - I essentially obtained a brand new card for around 8% of the retail price. Don’t get me wrong, I think this card is worth every bit of the HP $18,359.00 list price. However, the demand for second-hard cards of this caliber is very low so liquidators are pricing them ridiculously low compared to what they are really worth. I am the happy beneficiary.

    A new test is planned now for weekend of October 26 adding the new card as well as another SATA SSD as destination media. Along with this there will be a couple of configuration changes such as making the backup media all RAID-0 instead of RAID-1 (this is after all a test aimed at maximizing throughput). I am hoping to get my hands on some HP aux power cables before then to supplement the PCIE bus power as that may become an issue. The other big change will be that the database is being scaled up to 1TB and it will still be meaningful data. The SQL database includes a stored procedure that generates correlative aggregate pairings utilizing a variety of technical indicators between any of the 12,000 or so equities and indexes. The sample size for this process is currently less than 500 and that generates nearly 1 billion rows consuming 40GB of space.  Increasing the combinations can be used to exponentially multiply the amount of data and quickly bring it to over 1TB. My goal is to achieve 3.5 GBps backup throughput – moving 1TB will stress the wear leveling since at even 3.5 GBps it will take over 5 minutes for a 1TB database.

    image

    image

     

    image

    ]]>         Over 2 GBps SQL Server 2012 backup on 12-core server with 7 Fusion-IO Cards! https://docs.microsoft.com/archive/blogs/microsoftbob/over-2-gbps-sql-server-2012-backup-on-12-core-server-with-7-fusion-io-cards Thu, 18 Oct 2012 00:49:05 GMT https://blogs.msdn.microsoft.com/microsoftbob/2012/10/18/over-2-gbps-sql-server-2012-backup-on-12-core-server-with-7-fusion-io-cards/ I managed to squeeze 2 GBps (2192.207 MBps to be exact) out on 12 cores using a HP Server DL370 with 12 x 2.66 (12 MB Cache) cores, 108GB RAM, and 7 Fusion-IO Cards backing up from a database stored on 5 cards to backup files spread across 2 other cards, 1 Intel SSD, and 2 Samsung SSDs,. The receiving end was saturated. I could probably get closer to 3GBps if I balanced the load better by getting more SATA SSDs (although my built-in HP SATA III controller won’t handle very many SSDs) along with moving some more output to the cards currently being used by the database. I am going to rerun the test as soon as I get around to updating the firmware on the one Fusion-IO cards that is extremely out of date (will have to put it into a separate staging server and go through 3 firmware upgrades to get it working, guessing this card was never even used). These cards were all purchased off of EBAY for about 20 – 30% of retail price. With the exception of one card, they are all at 100% reserve and most only have a fraction of their expected life used, especially the 4 SLC Cards. The breakdown is:

    - 4 SLC 320GB Duos (8 drives)

    - 1 MLC 640GB Duo (2 drives)

    - 1 MLC 1.2 TB Duo (2 drives)

    - 1 MLC 320GB IoDrive

    - 1 SLC 320GB Duo awaiting firmware update.

     

    The database is using 3 SLC Duos and 2 MLC Duos providing 8 drives mostly in RAID-0 with a RAID-1 for the log. The combined read capability is about 3.75 GBps. The SATA SSDs collectively seem to provide about 400MBps collectively (the 2 Samsungs are in a RAID 1) and the 3 drives from the MLC 320GB IODrive and 2 160GB drives from the 320GB SLC add another 2.0 GBPS write capability.

    A couple of interesting things before getting to the images of the actual backup execution:

    1) I had to turn off backup compression in SQL to get best results. The issue is that the 12 cores start to max out at about 1.2 GBPs throughput with compression. However, without compression, the throughput seems more limited by the actual storage device. I almost doubled throughput on the same backup not using compression. Caveat here is that this is only because the storage is providing the data faster than the CPUs can compress it, this is only an issue with very high throughput storage and limited CPU resources, so do not take this as a best practice. I would love to run this on a 96 core with compression – we might be looking at 4 GBps instead of 2 GBps with more powerful CPU resources for this storage using backup compression.

    2) Tweaked the backup settings default values using more buffers, larger buffer size, etc. Also found that using more files was useful not only for ensuring granularity to balance the output across the devices correctly, but to maximize queue depth. I had to play with this some; Initial tests showed the backup stalling out because I was writing too much of the backup to the SATA SSDs. When I redistributed more to the IO cards, the backup utilized all of the devise all the way until the end rather than getting stuck on one. Something to keep in mind when doing striped backup, put enough files on the faster devices to even out the slower devices, it is striping based on data, not time and will happily keep writing to the slower device until getting all of the data on it even if the other files have all been buffered out to disk. Using mismatched media will also slow you down because I don’t think SQL will let the files get too far out of sync in the striping. My backup even with the less than optimal configuration still only took 2 minutes, so it was easy to see the lag at the end for the big files.

    3) These cards are running without any auxiliary power connectors just using the PCIE bus power. At the bottom, I included the output of the FIO-STATUS –A so you can see details of the cards used. I am still trying to figure out who to talk to at Fusion-IO or HP to get some of these, this was of minimal effect because the HP server does a great job of keeping the cards cooled and most of the activity is read, but with putting more cards on the receiving end, I suspect that the aux power will help.

    It is something to fathom – backing up a database of over 150GB in less than 1 1/2 minutes. This database contains billions of stock market transactions, not just a generated bunch of dummy data.

    Without further ado, here are a couple of screen snapshots, I really should put this on Youtube as it is incredible to watch this actually happen and watch the Resource monitor spike to over 5GBps read throughput at times. I think this is quite an accomplishment and shows the value of these cards. I’ve not found anybody else achieve this level of throughput at this low of a hardware cost with such a small server. This type of performance usually takes at least 48 cores and a 4-way processor in addition to the high-speed storage. This is just a 2-way Nephalem Xeon configuration. I am hoping to get this up to 3.5 GBps with the balancing of the cards better for write and read.  Of course if I had a couple of Duo-2s, who knows. The CPUs were only hitting about 20% with the compression turned off. With 4 newer Duos (the new cards require X8, the server has only 4 X8), this could potentially go to 6Gbps.

     

    clip_image002

    clip_image004

    Found 15 ioMemory devices in this system with 7 ioDrive Duos

    Fusion-io driver version: 3.1.1 build 181

    Adapter: Single Adapter

    HP 320GB MLC PCIe ioDrive for ProLiant Servers, Product Number:600279-B21, SN:XXXXX

    Pseudo Low-Profile ioDIMM Adapter, PN:00119200000

    External Power: NOT connected

    PCIe Bus voltage: avg 11.71V min 11.65V max 11.73V

    PCIe Bus current: avg 0.46A max 1.69A

    PCIe Bus power: avg 5.40W max 19.73W

    PCIe Power limit threshold: 24.75W

    PCIe slot available power: unavailable

    Connected ioMemory modules:

    fct2: Product Number:600279-B21

    fct2 Attached as 'fct2' (block device)

    HP ioDrive 320GB, Product Number:600279-B21

    HP ioDrive 320GB, PN:00214200302

    Located in slot 0 Center of Pseudo Low-Profile ioDIMM Adapter SN: XXXXX

    Powerloss protection: protected

    Last Power Monitor Incident: 33533 sec

    PCI:02:00.0, Slot Number:1

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178c

    Firmware v6.0.0, rev 107006 Public

    288.00 GBytes block device size

    Format: v500, 562500000 sectors of 512 bytes

    PCIe slot available power: unavailable

    Internal temperature: 43.31 degC, max 44.30 degC

    Internal voltage: avg 1.00V, max 1.02V

    Aux voltage: avg 2.46V, max 2.46V

    Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%

    Lifetime data volumes:

    Physical bytes written: 9,532,104,713,616

    Physical bytes read : 9,598,909,699,136

    RAM usage:

    Current: 126,144,192 bytes

    Peak : 126,951,232 bytes

    Adapter: Dual Adapter

    HP 320GB SLC PCIe ioDrive Duo for ProLiant Servers, Product Number:600281-B21, SN:59738

    ioDrive Duo HL, PN:00190000107

    External Power: NOT connected

    PCIe Bus voltage: avg 11.99V min 11.92V max 12.00V

    PCIe Bus current: avg 0.91A max 2.41A

    PCIe Bus power: avg 10.92W max 19.87W

    PCIe Power limit threshold: 24.75W

    PCIe slot available power: unavailable

    Connected ioMemory modules:

    fct16: Product Number:600281-B21

    fct17: Product Number:600281-B21

    fct16 Attached as 'fct16' (block device)

    HP ioDIMM 160GB, Product Number:600281-B21

    HP ioDIMM 160GB, PN:00277100101

    Located in slot 0 Upper of ioDrive Duo HL SN: XXXXX

    Powerloss protection: protected

    Last Power Monitor Incident: 33506 sec

    PCI:10:00.0, Slot Number:3

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178d

    Firmware v6.0.0, rev 107007 Public

    128.00 GBytes block device size

    Format: v500, 31250000 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 45.77 degC, max 46.26 degC

    Internal voltage: avg 1.03V, max 1.03V

    Aux voltage: avg 2.44V, max 2.44V

    Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%

    Lifetime data volumes:

    Physical bytes written: 799,340,235,816

    Physical bytes read : 1,393,505,907,368

    RAM usage:

    Current: 49,125,952 bytes

    Peak : 49,125,952 bytes

    fct17 Attached as 'fct17' (block device)

    HP ioDIMM 160GB, Product Number:600281-B21

    HP ioDIMM 160GB, PN:00277100101

    Located in slot 1 Lower of ioDrive Duo HL SN: XXXXX

    Powerloss protection: protected

    Last Power Monitor Incident: 33507 sec

    PCI:11:00.0, Slot Number:3

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178d

    Firmware v6.0.0, rev 107007 Public

    128.00 GBytes block device size

    Format: v500, 31250000 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 39.87 degC, max 40.85 degC

    Internal voltage: avg 1.01V, max 1.01V

    Aux voltage: avg 2.43V, max 2.43V

    Reserve space status: Healthy; Reserves: 93.90%, warn at 10.00%

    Lifetime data volumes:

    Physical bytes written: 497,387,016,664

    Physical bytes read : 1,115,459,630,528

    RAM usage:

    Current: 49,051,072 bytes

    Peak : 49,051,072 bytes

    Adapter: Dual Adapter

    HP 320GB SLC PCIe ioDrive Duo for ProLiant Servers, Product Number:600281-B21, SN: XXXXX

    ioDrive Duo HL, PN:00190000107

    External Power: NOT connected

    PCIe Bus voltage: avg 11.99V min 11.92V max 12.01V

    PCIe Bus current: avg 0.95A max 2.26A

    PCIe Bus power: avg 11.45W max 27.08W

    PCIe Power limit threshold: 24.75W

    PCIe slot available power: unavailable

    Connected ioMemory modules:

    fct21: Product Number:600281-B21

    fct22: Product Number:600281-B21

    fct21 Attached as 'fct21' (block device)

    HP ioDIMM 160GB, Product Number:600281-B21

    HP ioDIMM 160GB, PN:00277100103

    Located in slot 0 Upper of ioDrive Duo HL SN: XXXXX

    Powerloss protection: protected

    Last Power Monitor Incident: 33775 sec

    PCI:15:00.0, Slot Number:4

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178d

    Firmware v6.0.0, rev 107007 Public

    128.00 GBytes block device size

    Format: v500, 31250000 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 45.28 degC, max 45.77 degC

    Internal voltage: avg 1.03V, max 1.03V

    Aux voltage: avg 2.46V, max 2.46V

    Reserve space status: Healthy; Reserves: 93.89%, warn at 10.00%

    Lifetime data volumes:

    Physical bytes written: 2,399,961,878,072

    Physical bytes read : 3,486,203,149,520

    RAM usage:

    Current: 52,154,432 bytes

    Peak : 52,154,432 bytes

    fct22 Attached as 'fct22' (block device)

    HP ioDIMM 160GB, Product Number:600281-B21

    HP ioDIMM 160GB, PN:00277100103

    Located in slot 1 Lower of ioDrive Duo HL SN: XXXXX

    Powerloss protection: protected

    PCI:16:00.0, Slot Number:4

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178d

    Firmware v6.0.0, rev 107007 Public

    128.00 GBytes block device size

    Format: v500, 31250000 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 44.30 degC, max 45.77 degC

    Internal voltage: avg 1.00V, max 1.01V

    Aux voltage: avg 2.46V, max 2.46V

    Reserve space status: Healthy; Reserves: 93.90%, warn at 10.00%

    Lifetime data volumes:

    Physical bytes written: 2,509,265,675,328

    Physical bytes read : 3,186,755,344,040

    RAM usage:

    Current: 525,667,328 bytes

    Peak : 525,667,328 bytes

    Adapter: Dual Adapter

    HP 320GB SLC PCIe ioDrive Duo for ProLiant Servers, Product Number:600281-B21, SN: XXXXX

    ioDrive Duo HL, PN:00190000107

    External Power: NOT connected

    PCIe Bus voltage: min 7.88V max 12.21V

    PCIe Bus current: max 0.95A

    PCIe slot available power: unavailable

    Connected ioMemory modules:

    fct31: Product Number:600281-B21

    fct32: Product Number:600281-B21

    fct31 Status unknown: Driver is in MINIMAL MODE:

    The firmware on this device is not compatible with the currently installed version of the driver

    HP ioDIMM 160GB, Product Number:600281-B21

    !! ---> There are active errors or warnings on this device! Read below for details.

    HP ioDIMM 160GB, PN:00277100101

    Located in slot 0 Upper of ioDrive Duo HL SN: XXXXX

    Powerloss protection: not available

    PCI:1f:00.0, Slot Number:5

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178d

    Firmware v3.0.3, rev 43246 Public

    Geometry and capacity information not available.

    Format: not low-level formatted

    PCIe slot available power: unavailable

    Internal temperature: 43.80 degC, max 44.30 degC

    Internal voltage: avg 1.03V, max 1.03V

    Aux voltage: avg 2.46V, max 2.46V

    Lifetime data volumes:

    Physical bytes written: 0

    Physical bytes read : 0

    RAM usage:

    Current: 0 bytes

    Peak : 0 bytes

    ACTIVE WARNINGS:

    The ioMemory is currently running in a minimal state.

    fct32 Status unknown: Driver is in MINIMAL MODE:

    The firmware on this device is not compatible with the currently installed version of the driver

    HP ioDIMM 160GB, Product Number:600281-B21

    !! ---> There are active errors or warnings on this device! Read below for details.

    HP ioDIMM 160GB, PN:00277100101

    Located in slot 0 Upper of ioDrive Duo HL SN: XXXXX

    Powerloss protection: not available

    PCI:20:00.0, Slot Number:5

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178d

    Firmware v3.0.3, rev 43246 Public

    Geometry and capacity information not available.

    Format: not low-level formatted

    PCIe slot available power: unavailable

    Internal temperature: 42.82 degC, max 43.31 degC

    Internal voltage: avg 1.01V, max 1.01V

    Aux voltage: avg 2.45V, max 2.45V

    Lifetime data volumes:

    Physical bytes written: 0

    Physical bytes read : 0

    RAM usage:

    Current: 0 bytes

    Peak : 0 bytes

    ACTIVE WARNINGS:

    The ioMemory is currently running in a minimal state.

    Adapter: Dual Adapter

    HP 640GB MLC PCIe ioDrive Duo for ProLiant Servers, Product Number:600282-B21, SN: XXXXX

    ioDrive Duo HL, PN:00190000108

    External Power: NOT connected

    PCIe Bus voltage: avg 11.99V min 11.93V max 12.02V

    PCIe Bus current: avg 0.96A max 2.21A

    PCIe Bus power: avg 11.48W max 26.39W

    PCIe Power limit threshold: 24.75W

    PCIe slot available power: unavailable

    Connected ioMemory modules:

    fct36: Product Number:600282-B21

    fct37: Product Number:600282-B21

    fct36 Attached as 'fct36' (block device)

    HP ioDIMM 320GB, Product Number:600282-B21

    HP ioDIMM 320GB, PN:00277100201

    Located in slot 0 Upper of ioDrive Duo HL SN: XXXXX

    Powerloss protection: protected

    Last Power Monitor Incident: 33595 sec

    PCI:24:00.0, Slot Number:6

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178e

    Firmware v6.0.0, rev 107007 Public

    256.00 GBytes block device size

    Format: v500, 62500000 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 45.28 degC, max 45.77 degC

    Internal voltage: avg 1.03V, max 1.03V

    Aux voltage: avg 2.47V, max 2.48V

    Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%

    Lifetime data volumes:

    Physical bytes written: 3,071,101,924,128

    Physical bytes read : 3,407,184,536,048

    RAM usage:

    Current: 32,577,472 bytes

    Peak : 32,577,472 bytes

    fct37 Attached as 'fct37' (block device)

    HP ioDIMM 320GB, Product Number:600282-B21

    HP ioDIMM 320GB, PN:00277100201

    Located in slot 1 Lower of ioDrive Duo HL SN: XXXXX

    Powerloss protection: protected

    PCI:25:00.0, Slot Number:6

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178e

    Firmware v6.0.0, rev 107007 Public

    256.00 GBytes block device size

    Format: v500, 62500000 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 39.87 degC, max 40.36 degC

    Internal voltage: avg 1.01V, max 1.02V

    Aux voltage: avg 2.46V, max 2.47V

    Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%

    Lifetime data volumes:

    Physical bytes written: 2,380,119,569,280

    Physical bytes read : 2,736,125,677,440

    RAM usage:

    Current: 32,544,192 bytes

    Peak : 32,544,192 bytes

    Adapter: Dual Adapter

    HP 320GB SLC PCIe ioDrive Duo for ProLiant Servers, Product Number:600281-B21, SN: XXXXX

    ioDrive Duo HL, PN:00190000107

    External Power: NOT connected

    PCIe Bus voltage: avg 11.99V min 11.93V max 12.02V

    PCIe Bus current: avg 0.90A max 1.33A

    PCIe Bus power: avg 10.74W max 15.92W

    PCIe Power limit threshold: 24.75W

    PCIe slot available power: unavailable

    Connected ioMemory modules:

    fct41: Product Number:600281-B21

    fct42: Product Number:600281-B21

    fct41 Detached

    HP ioDIMM 160GB, Product Number:600281-B21

    HP ioDIMM 160GB, PN:00277100101

    Located in slot 0 Upper of ioDrive Duo HL SN: XXXXX

    Powerloss protection: protected

    PCI:29:00.0, Slot Number:7

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178d

    Firmware v6.0.0, rev 107007 Public

    144.00 GBytes block device size

    Format: v500, 35156250 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 40.85 degC, max 41.34 degC

    Internal voltage: avg 1.03V, max 1.03V

    Aux voltage: avg 2.46V, max 2.46V

    Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%

    Lifetime data volumes:

    Physical bytes written: 959,690,184

    Physical bytes read : 6,919,113,776

    RAM usage:

    Current: 10,920,000 bytes

    Peak : 10,920,000 bytes

    fct42 Detached

    HP ioDIMM 160GB, Product Number:600281-B21

    HP ioDIMM 160GB, PN:00277100101

    Located in slot 1 Lower of ioDrive Duo HL SN: XXXXX

    Powerloss protection: protected

    PCI:2a:00.0, Slot Number:7

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178d

    Firmware v6.0.0, rev 107007 Public

    144.00 GBytes block device size

    Format: v500, 35156250 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 42.33 degC, max 42.82 degC

    Internal voltage: avg 1.00V, max 1.00V

    Aux voltage: avg 2.45V, max 2.45V

    Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%

    Lifetime data volumes:

    Physical bytes written: 960,269,736

    Physical bytes read : 6,932,504,752

    RAM usage:

    Current: 10,920,000 bytes

    Peak : 10,920,000 bytes

    Adapter: Dual Adapter

    HP 320GB SLC PCIe ioDrive Duo for ProLiant Servers, Product Number:600281-B21, SN: XXXXX

    ioDrive Duo HL, PN:00190000107

    External Power: NOT connected

    PCIe Bus voltage: avg 11.99V min 11.92V max 12.02V

    PCIe Bus current: avg 0.97A max 2.31A

    PCIe Bus power: avg 11.58W max 27.61W

    PCIe Power limit threshold: 24.75W

    PCIe slot available power: unavailable

    Connected ioMemory modules:

    fct46: Product Number:600281-B21

    fct47: Product Number:600281-B21

    fct46 Attached as 'fct46' (block device)

    HP ioDIMM 160GB, Product Number:600281-B21

    HP ioDIMM 160GB, PN:00277100101

    Located in slot 0 Upper of ioDrive Duo HL SN: XXXXX

    Powerloss protection: protected

    Last Power Monitor Incident: 33713 sec

    PCI:2e:00.0, Slot Number:8

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178d

    Firmware v6.0.0, rev 107007 Public

    128.00 GBytes block device size

    Format: v500, 31250000 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 44.30 degC, max 44.79 degC

    Internal voltage: avg 1.02V, max 1.03V

    Aux voltage: avg 2.47V, max 2.47V

    Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%

    Lifetime data volumes:

    Physical bytes written: 661,125,209,496

    Physical bytes read : 1,182,061,604,688

    RAM usage:

    Current: 525,667,840 bytes

    Peak : 525,667,840 bytes

    fct47 Attached as 'fct47' (block device)

    HP ioDIMM 160GB, Product Number:600281-B21

    HP ioDIMM 160GB, PN:00277100101

    Located in slot 1 Lower of ioDrive Duo HL SN:104395

    Powerloss protection: protected

    PCI:2f:00.0, Slot Number:8

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:178d

    Firmware v6.0.0, rev 107007 Public

    128.00 GBytes block device size

    Format: v500, 31250000 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 41.83 degC, max 42.33 degC

    Internal voltage: avg 1.00V, max 1.01V

    Aux voltage: avg 2.46V, max 2.47V

    Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%

    Lifetime data volumes:

    Physical bytes written: 612,235,065,752

    Physical bytes read : 1,107,146,274,888

    RAM usage:

    Current: 525,666,880 bytes

    Peak : 525,666,880 bytes

    Adapter: Dual Adapter

    HP 1280GB MLC PCIe ioDrive Duo for ProLiant Servers, Product Number:641027-B21, SN: XXXXX

    ioDrive Duo HL, PN:00190000108

    External Power: NOT connected

    PCIe Bus voltage: avg 12.00V min 11.93V max 12.02V

    PCIe Bus current: avg 0.93A max 1.87A

    PCIe Bus power: avg 11.17W max 22.36W

    PCIe Power limit threshold: 24.75W

    PCIe slot available power: unavailable

    Connected ioMemory modules:

    fct51: Product Number:641027-B21

    fct52: Product Number:641027-B21

    fct51 Attached as 'fct51' (block device)

    ioDIMM 640, SN: XXXXX

    ioDIMM 640, PN:00277100605

    Located in slot 0 Upper of ioDrive Duo HL SN: XXXXX

    Powerloss protection: protected

    Last Power Monitor Incident: 33703 sec

    PCI:33:00.0, Slot Number:9

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:176f

    Firmware v6.0.0, rev 107007 Public

    512.00 GBytes block device size

    Format: v500, 125000000 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 41.34 degC, max 42.33 degC

    Internal voltage: avg 1.02V, max 1.03V

    Aux voltage: avg 2.47V, max 2.47V

    Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%

    Rated PBW: 10.00 PB, 98.96% remaining

    Lifetime data volumes:

    Physical bytes written: 103,651,963,293,456

    Physical bytes read : 127,186,509,181,944

    RAM usage:

    Current: 53,448,192 bytes

    Peak : 53,448,192 bytes

    fct52 Attached as 'fct52' (block device)

    ioDIMM 640, SN: XXXXX

    ioDIMM 640, PN:00277100605

    Located in slot 1 Lower of ioDrive Duo HL SN: XXXXX

    Powerloss protection: protected

    PCI:34:00.0, Slot Number:9

    Vendor:1aed, Device:1005, Sub vendor:103c, Sub device:176f

    Firmware v6.0.0, rev 107007 Public

    512.00 GBytes block device size

    Format: v500, 125000000 sectors of 4096 bytes

    PCIe slot available power: unavailable

    Internal temperature: 38.39 degC, max 38.88 degC

    Internal voltage: avg 1.00V, max 1.01V

    Aux voltage: avg 2.46V, max 2.46V

    Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%

    Rated PBW: 10.00 PB, 98.98% remaining

    Lifetime data volumes:

    Physical bytes written: 102,299,237,920,520

    Physical bytes read : 126,266,803,141,400

    RAM usage:

    Current: 53,489,792 bytes

    Peak : 53,489,792 bytes

    ]]>         Software Quality - 4 Key facets (From Linked-in) https://docs.microsoft.com/archive/blogs/microsoftbob/software-quality-4-key-facets-from-linked-in Sun, 29 Apr 2012 02:51:53 GMT https://blogs.msdn.microsoft.com/microsoftbob/2012/04/29/software-quality-4-key-facets-from-linked-in/ I have become active in the Linked-in Community, especially regarding predictive analytics, as there are many very good discussion groups.

    So, to save effort and still get a couple of hours of sleep, a lot of my posting here will come from there.

    The below is not a predictive analytics post, but an answer to question on how to ensure software quality beyond simply meeting the customer requirements that I thought I would share. There are other aspects besides Extensibility, Scalability, Traceability, and Reliability, but these are four pretty good starting pillars.

    Software quality assurance requires disciplined software
    development, risk management, planning, and ongoing evaluation. Many systems
    start out as satisfactory to the users, but are illusions. One analogy is that
    you can build a house without a satisfactory foundation and it might be fine at
    first until it starts falling down and there is no way to repair it without
    rebuilding.

    The same thing often happens in software development. Many
    times, what is intended as a throw-away solution for demonstration purposes but
    lacks the robust architecture needed to support growing the systems are
    implemented as production systems due to time or cost deadlines. However, the
    cost is paid later.

    Quality software should possess the following capabilities

    1) The ability to add more stress in terms of users,
    transactions, etc. by simply adding more hardware without software changes.
    This is known as scalability. Some design approaches cannot scale beyond a
    certain level and then fail.

    2) The ability to extend functionality without redesigning
    the system. This requires careful planning in creating a well-structured
    (normalized) database, developing components in a modular way so that they can
    be re-used easily, and creating the system in logical layers for presentation,
    business rules, data access, etc. This can be thought of as extensibility.

    3) Building a testing and documentation framework into the
    system rather than as an afterthought (ensuring reliability). Many systems go
    into production inadequately tested resulting in far greater cost than if they
    had been tested thoroughly. Documentation goes along with this in describing
    how the system is expected to operate, what defines success, and the methods
    for verifying that the system works effectively.

    4) An effective change management system must be used to
    track all changes and help with the planning (Traceability). There are some
    very good products available to help with this including Microsoft Team
    Foundation Server.

    A key method for achieving software quality is the use of an
    iterative development cycle and test-driven development to regularly create
    value for the customer and verify the software is meeting the requirements.
    This fosters a framework for continuous improvement. It is usually impossible
    to foresee all the requirements for a system and technology continually enables
    new functionality, so incremental delivery is the most cost-effective long-term
    strategy for not only customer satisfaction but ongoing software quality.

    There are more formal definitions of software quality, but
    reliability, scalability, extensibility, and traceability are key elements.
    There are frameworks and approaches defined to help instill accountability into
    the software development process to help achieve software quality including
    Capability Maturity Model (CMM), Microsoft Solutions Framework (MSF), and
    Agile/Scrum development approaches. The links for these are below:

    Links:

    ]]>         Memory via PCIE-SSD as the authoritative data source https://docs.microsoft.com/archive/blogs/microsoftbob/memory-via-pcie-ssd-as-the-authoritative-data-source Mon, 26 Mar 2012 11:34:00 GMT https://blogs.msdn.microsoft.com/microsoftbob/2012/03/26/memory-via-pcie-ssd-as-the-authoritative-data-source/ i've blogged a few times about Fusion-IO devices and my experiences with SSD. I've been thinking about the implications of large in-memory databases that retain persistence and transactional capabilities. Turns out the game may be changing long-term to an in-memory database model forscalability. I.e. Fusion-IO demonstrated technology that extends memory to Fusion-IO devices and supports up to 19.2 TB mapped as permanent memory across an 8-way redundant network of commodity hardware using a technology known as auto-commit-memory – see https://www.fusionio.com/press-releases/fusion-io-breaks-one-billion-iops-barrier/. Via high-speed PCIE, the line between storage and memory is crossed, memory = storage and storage = memory.

    Huge memory that is both partitioned and replicated with hundreds of servers in a cloud and an abstraction layer for transforming queries via web-services to the cloud and yu’ve solved the scalability problem. Easily, you could support petabytes of storage between hundreds of servers that together have all the memory in memory at the same time with redundancy and persistence to boot.

    The authoritative data for future RDBMS looks to be in memory-data accessed via the cloud with a controller layer to parlay queries.  SQL databases become merely snapshotsin time. If you factor the growth rate in flash density, PCIE 3.0 with bandwidth higher than DDR3 and the approach to commoditize PCIE in the same way 2.5 SAS slots are (see the Dell R720 offering with hot-swap PCIE), then we may be easily looking at memory-mapped 100TB machines in the next couple of years. That along with controller-layer and partitioning schemes to divide up large datasets among peer servers of the and you have scalability through an interface that runs the query against in-memory representations of tables/indexes rather than on-disk versions with peer-to-peer memory mirroring through messaging rather directed from the stored databases.

    I'm still getting upto speed on HADOOP, etc, but memory shared between multiple servers seamlessly as authoritative is probably a key enabler.

    Sounds pretty radical, but the times – They are a-Changin – Bob Dylan –- https://www.youtube.com/watch?v=vCWdCKPtnYE

    Bob

    ]]>         Boolean KMAP reduction code released after 23 years! https://docs.microsoft.com/archive/blogs/microsoftbob/boolean-kmap-reduction-code-released-after-23-years Fri, 24 Feb 2012 23:31:00 GMT https://blogs.msdn.microsoft.com/microsoftbob/2012/02/24/boolean-kmap-reduction-code-released-after-23-years/ Yes, I know its been a while since I have posted anything and way past due to post my update on my fusion-io card testing (I now have 6 memory devices in my R710 - 2 cards are duo). I will hopefully get to that in the next couple of weeks as I am using all my spare time to try to finish my dissertation that has been in the work for a few years now, in fact the dissertation is focused on a problem I first became enamored with solving over 25 years ago.

    My community involvement metric needs to improve, so am posting some code from the archives...The code is slightly out of date, completed in December, 1988. I was going to wait until the 25th anniversary of the program, but just could not resist.

    The attached listing is how we used to code in the prehistoric era before cell phones when PCs were limited to 640K of RAM. The days before indoor plumbing and paved roads when most humans still lived in caves and dinosaurs roamed the earth…

    The .txt file attachd contains the compiler listing of a COBOL program I wrote for a school project that could find the optimal expression for up to 1024 Karnaugh expressions with up to 10 min-terms including “don’t cares” in a couple minutes. That was on a DEC VAX “mini-computer” about the size of a refrigerator with as I recall about 3MB of RAM and processed at 1 MIPs, or about 40 times slower than the average cell phone. Fortunately, VAX/VMS included virtual memory management - the size of the compiled image was probably close to the size of total memory on the machine. Here's an article that explains Karnaugh expressions:

    Here’s a picture of what it looked like: https://hampage.hu/vax/kepek/sokvax.jpg

    The instructor did not appreciate this project and gave me a “C”. He did not believe it could work and had argued with me that computers could not perform logic, they could only do calculations, therefore no software program could perform a Karnaugh map reduction. I think the fact that I chose to do this in COBOL didn’t help either, since as many other academians he expressed disdain for the language.

    And yes, the code actually worked. You can read about Karnaugh maps at wikipedia - https://en.wikipedia.org/wiki/Karnaugh_map. The code utilizes aspects from Quine McCluskey method.

     Bob

    bool.lis.txt

    ]]>