After nearly nine years at Microsoft, I’ve chosen to move on.

I’ve accepted a position at Fusion-io—now a SanDisk company—as Worldwide SQL Server Solutions Architect where I’ll be evangelizing, documenting best practices, collaborating directly with the Windows & SQL product teams & other partners, doing cutting-edge performance work, & of course flipping the /faster bit—all things about which I’m passionate—& doing so with the full faith & support of the best enterprise flash company on the planet.  I collaborated closely with several of these fine folks while managing the SQL CAT Customer Lab.  Like me, they’re passionate about performance, defining engineering discipline, & delighting the customer.  It’s a fantastic fit.

Why “Hello, World! Encore Edition”?  My very first MSDN post, “Hello, World!” reflects aspirations that remain true today.  (It also includes an hilarious excerpt from Scott Adams on “the why” of blogging.)  So this post isn’t good-bye¸ it’s simply a transition.

Contact: Blog & Twitter

My primary blog will be here: http://sqlblog.com/blogs/jimmy_may where I’ll continue to post tips, tricks, speaking events, & of course, insights on aforementioned /faster bit.  That site, hosted by MVP Adam Machanic, features an awesome roster of talent that I’m honored to be a part of.

I invite you to look me up on Facebook as well as Twitter: @AspiringGeek.

Here’s my email:  SELECT REVERSE('moc.evil@keeggniripsa')

Changing the World

Anyone who’s read my blog or heard me speak know I’ve liberated my motto from @GapingVoid

Change the world is exactly what I’ve tried to do—& will continue to do—to make it a better place, to best serve our customers.

I had lots of help, working with lots of great engineers & lots of great customers—lots of great people.

Along the way I authored one white paper and was named contributor or technical reviewer on almost two dozen others.  I had the privilege of editing or contributing to several books, including the career- & life-changing Getting Results the Agile Way: A Personal Results System for Work & Life by MS Principal PM J.D. Meier (Sources of Insight | 30 Days of Getting Results | wiki | amazon).

 
Test-drive the book on how to make the most of work and life.
Read Getting Results the Agile Way for free online.

I spoke at the PASS International Summit five times, TechReady seven times, & at dozens of user groups across the country & some outside.

Here’s one of favorite moments on stage, demonstrating how columnstore turns conventional row store on its head with my #SQLWingMan, Shahry Hashemi aka @dsfnet (photos courtesy of @GEEQL).

As a consultant, I led over 60 on-site customer engagements around the world.  As a SQL CAT PM & manager of the Customer Lab, we shepherded over 100 customer engagements.  With a lot of help from my team, we architected the Customer Lab to host parallel engagements, remote engagements, streamlined our onboarding processes, paved the way for Cloud engagements, upgraded our network & I/O infrastructures, & enhanced our collaboration with several partners resulting in the acquisition of dozens of the latest-&-greatest servers & storage devices.  At MSIT I was part of the team troubleshooting performance issues & establishing best practices during our massive migration to Azure.

The Roles

Here’s a summary of my exciting roles while at Microsoft:

• Microsoft Consulting Services, Senior Consultant
• Microsoft A.C.E. Performance Team, Principal Consultant
• The SQL Server Customer Advisory Team—SQL CAT, Senior Program Manager
• Microsoft Information Technology—MSIT, Principal Architect

The Training: Microsoft Certified Master #MCM4Life

Just as SQL CAT was in so many ways a dream job, Microsoft Certified Masters training was the highlight of my professional training.  I prepped for months, reading every single page of the required reading—Kimberly Tripp later stating I was “The One” who’d done so. ;-)  At that time the cert had a required three-week on-site component during which, one after another, we were instructed by the best SQL Server trainers on the planet.  Just as importantly, I was privileged to sit alongside two dozen peers, all of whom remain friends.  On Labor Day weekend 2012, Microsoft Learning unceremoniously without warning terminated the program.  Many of us hope the program is resuscitated in a meaningful way.  In the meantime, MCM training was in so many ways an incredible, special, unique experience.

    
Certification logo courtesy of MSL combined with lots of hard work.
Gang tat courtesy of Robert Davis, yo. 

The People

Attempting to thank all those who’ve supported, encouraged, & championed my career reminds me of the dilemma faced by Academy Award winners—it’s impossible to properly acknowledge everyone in the allotted time, & there’s the risk of leaving out many who richly deserve credit.  But I’m going to try anyway.

If I were to try enumerating my myriad community colleagues, allies, & friends outside of Microsoft, the list would be unmanageable.

Yet I must recognize those in Microsoft leadership positions without whom my success would not have been possible:

• Eddie Lau & Irfan Chaudhry of A.C.E. Perf
• Mark Souza & Lindsey Allen of SQL CAT

And here (in no special order) are just a few of the current & former Microsoft staff who’ve been invaluable:

• My “Master Mind”: J.D. Meier, Alik Levin, Rob Boucher
• Especially Special Persons: Marilyn Grant, Janelle Aberle
• Mentors: Ty Moore, James Day
• MCS Mentors: Joe Sack & Kate Baroni
• SQL CAT: Shaun Tinline, Sanjay Mishra, Mike Ruthruff, Thomas Kejser, Denny Lee, Cindy Gross, Lara Rubbelke, Kathy MacDonald, Chuck Heinzelman, Ewan Fairweather, Mike Weiner, Tom Davidson, Regina Jones, Mike Anderson
• MCMs: Jens Suessmeyer, Bertil Syamken, Jose Barrios, Robert Davis, Argenis Fernandez, Kalyan Yella, Tracey Jordan, Cris Benge
• MCM Trainers: Kalen Delaney, Paul Randal, Kimberly Tripp, Greg Low, Adam Machanic
• MSIT: Chris Lundquist, James Beeson, Vitaliy Konev, Casie Owen, Brian Walters, Ahmad Mahdi, Dale Hirt, Rob Beddard
• PFEs: Gennady Kostinsky, Clint Huffman, Shane Creamer, Ken Brumfield, Robert Smith, Kaitlin Goddard
• SQL BI: Kay Unkroth
• SQL PG:  Sunil Agarwal, Eric Hanson, Susan Price, Kevin Farlee, Luis Carlos Vargas Herring
• Others: Buck Woody, Pablo Brontvain, Steven Schneider, Brian Raymer, Steven Wort, Bruce Worthington, Dandy Weyn, Cephas Lin, Matthew Robertshaw, Mark Pohto, Bob Roudebush

Living Life to the varchar(max)

Thank you, everyone—inside & outside of Microsoft—for your help & friendship during my tenure. I couldn’t’ve done it without you.  Our journey continues, & I look forward to exploring the road of happy destiny together with all of you. 

In the meantime, here’s to living life to the varchar(max)!

Jimmy May, Aspiring Geek

Register, see the schedule, or see the event home page on the SQL Saturday site.  I’ll look forward to seeing you here:

Microsoft MTC
5404 Wisconsin Ave
Chevy Chase, MD 20815

Join me & an All-Star cast of speakers including MVPs such as Reeves Smith, Joey D’Antoni, Allen White, Jessica Moss, Grant Fitchey, Steve Jones, Randy Knight, Jason Brimhall, & Wayne Sheffield.  Other luminaries include shiny new Microsoft PFE Theresa Iserman & Consultant Ayman El-Ghazali.

To The Cloud, Infinity, & Beyond: Top 10 Lessons Learned at MSIT

I’ll be joined by my colleague, Principal Architect Rob Beddard a brilliant, no-nonsense engineer who’s shepherding much of the MSIT effort.

Much of the content may not strike you as profound, e.g., you’ll know all about planning, or about the I/O challenges.  Yet I know from surveying my peers that many of us have gaps.  I’m confident you’ll walk away with new insights.  Here’re details.

Session Summary:  
Microsoft IT's Cloud strategy involves migrating hundreds of applications to Azure.  The scale on which it is being done may be unprecedented. Lessons Learned are numerous. This session exposes the most important—& painful—ones.

Dear Readers, think about that—we’re migrating literally thousands of apps.

Here’s additional info about the session.

Session Goal 1: Understand that forklifting applications to the Cloud has unexpected challenges.
Session Goal 2: Understand that many fundamental best practices which have served us well on-prem remain important—& now we have new nuances to master.
Session Goal 3: Understand the value of leveraging MSIT's lessons learned has the potential to save an organization weeks or months of time & hundreds of thousands, perhaps millions of dollars.

This is an exciting topic the work for which I'm intimately involved. I'm extremely enthusiastic about sharing the fruits of our efforts with the Community.  Join us!

Columnstore Indexes in SQL Server 2014: Flipping the DW /faster Bit

If you’ve not been in a crypt since last All Hallows Eve, you likely know this is one of my staples. 

Register, see the schedule, or see the event home page on the SQL Saturday site.  I’ll look forward to seeing you here:

Mittleman Community Center
6651 SW Capitol Highway
Portland, OR 97219

Kudos to Arnie Rowland (@ArnieRowland), Paul Turley (@Paul_Turley), Theresa Iserman (@TheresaIserman), Vern Rabe, Rob Boek (@robboek), & the rest of the Oregon SQL Users Group (@osqld) Leadership Team for their superb organizational efforts.

SQL Saturday Oregon All-Stars: Speaker Tom Roush (@GEEQL)
flanked by Leadership Team members Theresa Iserman (also a speaker) & MVP Arnie Rowland.

My talks are:

To the Cloud, Infinity, & Beyond: Top 10 Lessons Learned at MSIT

&

Columnstore Indexes in SQL Server 2014: Flipping the DW /faster Bit

The latter is one of my staples.  The former is a new presentation, a preview of my forthcoming delivery for PASS Summit14.

Register, see the schedule, or see the event home page on the SQL Saturday site.  I’ll look forward to seeing you here:

Spencer Fox Eccles Business Building
1768 Campus Center Drive
Salt Lake City, UT 84112

Kudos to Pat Wright (blog | @sqlasylum) & crew for their crazy efforts (pardon the pun) in coordinating the event.

TJay Belt (@tjaybelt), Andrea Alred (@RoyalSQL), & Ben Miller (@DBADuck), keep your peepers peeled—I’m on the way!

Where in the World is AdventureWorks?

Recently, SQL Community feedback from twitter prompted me to look in vain for SQL Server 2014 versions of the AdventureWorks sample databases we’ve all grown to know & love.

I searched Codeplex, then used the bing & even the google in an effort to locate them, yet all I could find were samples on different sites highlighting specific technologies, an incomplete collection inconsistent with the experience we users had learned to expect.  I began pinging internally & learned that an update to AdventureWorks wasn’t even on the road map. 

Fortunately, SQL Marketing manager Luis Daniel Soto Maldonado (t) lent a sympathetic ear & got the update ball rolling; his direct report Darmodi Komo recently announced the release of the shiny new sample databases for OLTP, DW, Tabular, and Multidimensional models to supplement the extant In-Memory OLTP sample DB. 

What Success Looks Like

In my correspondence with the team, here’s how I defined success:

1. Sample AdventureWorks DBs hosted on Codeplex showcasing SQL Server 2014’s latest-&-greatest features, including: 

• In-Memory OLTP (aka Hekaton)
• Clustered Columnstore
• Online Operations
• Resource Governor IO

2. Where it makes sense to do so, consolidate the DBs (e.g., showcasing Columnstore likely involves a separate DW DB)

3. Documentation to support experimenting with these features

As Microsoft Senior SDE Bonnie Feinberg (b) stated, “I think it would be great to see an AdventureWorks for SQL 2014.  It would be super helpful for third-party book authors and trainers.  It also provides a common way to share examples in blog posts and forum discussions, for example.” 

Exactly.  We’ve established a rich & robust tradition of sample databases on Codeplex.  This is what our community & our customers expect.  The prompt response achieves what we all aim to do, i.e., manifests the Service Design Engineering mantra of “delighting the customer”.  Kudos to Luis’s team in SQL Server Marketing & Kevin Liu’s team in SQL Server Engineering for doing so.

Download AdventureWorks 2014

Download your copies of SQL Server 2014 AdventureWorks sample databases here.

This is the second in a series of posts documenting big wins encountered using columnstore indexes in SQL Server 2012 & 2014.  Many of these can be found in my big deck along with details such as internals, best practices, caveats, etc.  The purpose of sharing the case studies in this context is to provide an easy-to-consume quick-reference alternative. See also Columnstore Case Study #1: MSIT SONAR Aggregations

Why Columnstore?

As stated previously, If we’re looking for a subset of columns from one or a few rows, given the right indexes, SQL Server can do a superlative job of providing an answer. If we’re asking a question which by design needs to hit lots of rows—DW, reporting, aggregations, grouping, scans, etc., SQL Server has never had a good mechanism—until columnstore.

Columnstore indexes were introduced in SQL Server 2012. However, they're still largely unknown. Some adoption blockers existed; yet columnstore was nonetheless a game changer for many apps.  In SQL Server 2014, potential blockers have been largely removed & they're going to profoundly change the way we interact with our data.  The purpose of this series is to share the performance benefits of columnstore & documenting columnstore is a compelling reason to upgrade to SQL Server 2014.

The Customer

DevCon Security provides home & business security services & has been in business for 135 years. I met DevCon personnel while speaking to the Utah County SQL User Group on 20 February 2012. (Thanks to TJ Belt (b|@tjaybelt) & Ben Miller (b|@DBADuck) for the invitation which serendipitously coincided with the height of ski season.)

The App: DevCon Security Reporting: Optimized & Ad Hoc Queries

DevCon users interrogate a SQL Server 2012 Analysis Services cube via SSRS. In addition, the SQL Server 2012 relational back end is the target of ad hoc queries; this DW back end is refreshed nightly during a brief maintenance window via conventional table partition switching.

SSRS, SSAS, & MDX

Conventional relational structures were unable to provide adequate performance for user interaction for the SSRS reports. An SSAS solution was implemented requiring personnel to ramp up technically, including learning enough MDX to satisfy requirements.

Ad Hoc Queries

Even though the fact table is relatively small—only 22 million rows & 33GB—the table was a typical DW table in terms of its width: 137 columns, any of which could be the target of ad hoc interrogation. As is common in DW reporting scenarios such as this, it is often nearly to optimize for such queries using conventional indexing.

DevCon DBAs & developers attended PASS 2012 & were introduced to the marvels of columnstore in a session presented by Klaus Aschenbrenner (b|@Aschenbrenner)

The Details

Classic vs. columnstore before-&-after metrics are impressive.

     

Scenario	Conventional Structures	Columnstore	Δ
SSRS via SSAS	10 - 12 seconds	1 second	>10x
Ad Hoc	5-7 minutes (300 - 420 seconds)	1 - 2 seconds	>100x

Here are two charts characterizing this data graphically.  The first is a linear representation of Report Duration (in seconds) for Conventional Structures vs. Columnstore Indexes. 

As is so often the case when we chart such significant deltas, the linear scale doesn’t expose some the dramatically improved values corresponding to the columnstore metrics.  Just to make it fair here’s the same data represented logarithmically; yet even here the values corresponding to 1 –2 seconds aren’t visible. 

The Wins

1. Performance: Even prior to columnstore implementation, at 10 - 12 seconds canned report performance against the SSAS cube was tolerable. Yet the 1 second performance afterward is clearly better. As significant as that is, imagine the user experience re: ad hoc interrogation. The difference between several minutes vs. one or two seconds is a game changer, literally changing the way users interact with their data—no mental context switching, no wondering when the results will appear, no preoccupation with the spinning mind-numbing hurry-up-&-wait indicators.  As we’ve commonly found elsewhere, columnstore indexes here provided performance improvements of one, two, or more orders of magnitude.
2. Simplified Infrastructure: Because in this case a nonclustered columnstore index on a conventional DW table was faster than an Analysis Services cube, the entire SSAS infrastructure was rendered superfluous & was retired.
3. PASS Rocks: Once again, the value of attending PASS is proven out. The trip to Charlotte combined with eager & enquiring minds let directly to this success story. Find out more about the next PASS Summit here, hosted this year in Seattle on November 4 - 7, 2014.

DevCon BI Team Lead Nathan Allan provided this unsolicited feedback:

“What we found was pretty awesome. It has been a game changer for us in terms of the flexibility we can offer people that would like to get to the data in different ways.”

Summary

For DW, reports, & other BI workloads, columnstore often provides significant performance enhancements relative to conventional indexing.  I have documented here, the second in a series of reports on columnstore implementations, results from DevCon Security, a live customer production app for which performance increased by factors of from 10x to 100x for all report queries, including canned queries as well as reducing time for results for ad hoc queries from 5 - 7 minutes to 1 - 2 seconds. As a result of columnstore performance, the customer retired their SSAS infrastructure.

I invite you to consider leveraging columnstore in your own environment. Let me know if you have any questions.

This is the first in a series of posts documenting big wins encountered using columnstore indexes in SQL Server 2012 & 2014.  Many of these can be found in this deck along with details such as internals, best practices, caveats, etc.  The purpose of sharing the case studies in this context is to provide an easy-to-consume quick-reference alternative.

Why Columnstore?

If we’re looking for a subset of columns from one or a few rows, given the right indexes, SQL Server can do a superlative job of providing an answer. If we’re asking a question which by design needs to hit lots of rows—DW, reporting, aggregations, grouping, scans, etc., SQL Server has never had a good mechanism—until columnstore.

Columnstore indexes were introduced in SQL Server 2012. However, they're still largely unknown. Some adoption blockers existed; yet columnstore was nonetheless a game changer for many apps.  In SQL Server 2014, potential blockers have been largely removed & they're going to profoundly change the way we interact with our data.  The purpose of this series is to share the performance benefits of columnstore & documenting columnstore is a compelling reason to upgrade to SQL Server 2014.

App: MSIT SONAR Aggregations

At MSIT, performance & configuration data is captured by SCOM. We archive much of the data in a partitioned data warehouse table in SQL Server 2012 for reporting via an application called SONAR.  By definition, this is a primary use case for columnstore—report queries requiring aggregation over large numbers of rows.  New data is refreshed each night by an automated table partitioning mechanism—a best practices scenario for columnstore.

The Win

Compared to performance using classic indexing which resulted in the expected query plan selection including partition elimination vs. SQL Server 2012 nonclustered columnstore, query performance increased significantly.  Logical reads were reduced by over a factor of 50; both CPU & duration improved by factors of 20 or more.  Other than creating the columnstore index, no special modifications or tweaks to the app or databases schema were necessary to achieve the performance improvements.  Existing nonclustered indexes were rendered superfluous & were deleted, thus mitigating maintenance challenges such as defragging as well as conserving disk capacity.

Details

The table provides the raw data & summarizes the performance deltas.

	Logical Reads (8K pages)	CPU (ms)	Durn (ms)
Columnstore	160,323	20,360	9,786
Conventional Table & Indexes	9,053,423	549,608	193,903
Δ	x56	x27	x20

The charts provide additional perspective of this data.  "Conventional vs. Columnstore Metrics" document the raw data.  Note on this linear display the magnitude of the conventional index performance vs. columnstore. 

The “Metrics (Δ)” chart expresses these values as a ratio.

Summary

For DW, reports, & other BI workloads, columnstore often provides significant performance enhancements relative to conventional indexing.  I have documented here, the first in a series of reports on columnstore implementations, results from an initial implementation at MSIT in which logical reads were reduced by over a factor of 50; both CPU & duration improved by factors of 20 or more.  Subsequent features in this series document performance enhancements that are even more significant. 

Last year MSIT’s People Aspirations team asked me to share my Microsoft certification experiences. The team adopted a Bruce Lee meme which I weaved into my deck.

This week the PowerPoint team deployed a bug fix enabling embedded YouTube clips (the Insert > Video command has been extant for years, but it didn’t work). I celebrated by retrofitting the deck which I've attached for your consumption—& hopefully, your inspiration.

N00b > MOUS > MCP > MCITP > MCSE > MCDBA > MCSA > #SQLMCM > MCSM > #MCM4Life

I'm a big fan of certification. Without the knowledge my exam prep has provided, my career path would have been quite different.  I have a former peer who’s still working help desk (not that there’s anything wrong with that…).  My path included being an essential part of scaling up a startup (which has since IPO'd) to a solo consulting practice to consulting at Microsoft to the hallowed halls of SQL CAT & now Architect at MSIT—my path plain-&-simply would have been neither as intrinsically rewarding nor as financially lucrative.  Just as important are the relationships I’ve built—peers here at Microsoft, awesome customers, lifelong friends, & my fantastic tweeps.

My experience is evidence that certification combined with enthusiasm & application yields results.  Judging by my peers, my experience isn’t unique.

What initially moved me to get certified eludes me today, but the journey started with a Windows 95 cert, followed by my first SQL cert—SQL Server 7 Administration.  In 2000, people were so desperate for SQL “expertise” that if you could fog a mirror & spell S-Q-L—or better yet know how to pronounce it—you were hired. An actual certification—oh my!—was enough for my transition from Help Desk at a financial org to almost doubling my salary(!) as a Senior DBA at an Internet startup (my enthusiasm didn't hurt, either).

As most of you know, I'm a Microsoft Certified Master MCM:SQL.  Earning that cert was one of the most challenging things I’ve done.

SQL Ranger Training: In Building 40, no one can hear you scream.
    —Per Farny, former Microsoft MCA Program Director

Yes, last Labor Day weekend MS Learning’s decision went viral when under the cover of darkness unilaterally, inexplicably, & suddenly dropped their advanced cert programs (tweet|blog|blog|blog|story|story|story, etc…).  Yet that's not stopped me from continuing my journey.  In fact, even though owing to a circumstance of the program retirement I’m an #MCM4Life, I've earned four more SQL certs—count ‘em, four—in the last year.

¡Yo! Image courtesy of Robert Davis (b|t): http://twitter.yfrog.com/g0y76rp?sa=0

I'm unlikely ever again to score a 100% like I did on a SQL 2005 exam. Long gone are the days I could take these tests cold.  Now I actually have to study. They're a lot of work, yet doing so is an exercise in Stephen Covey's 7th Habit: Sharpen the Saw.

Don’t Marinate in Your Own Stench

Though there are lots of self-didactic opportunities, training can be expensive.  How to pay for it?  Perhaps your employer can subsidize all or part.  Those of us with employers can negotiate training time & dollars in our annual reviews.  When taking a new position, if time & expenses for the PASS Summit aren’t subsidized, it’s a non-starter for me.  Be ready, willing, & able to share your learnings with your colleagues!

The only thing worse than training employees and having them leave is not training employees and having them stay.
  —Zig Ziglar

There’s another option.  Economist Walter Williams suggests that investing in our own intellectual capital is the best investment we can make.  Early in my career I forked out $10,000 out-of-pocket for a year-long one-night-per-week MCSE training regimen.  It was one of the best investments I’ve ever made.  Learn more in my deck.  Elevate your game each-&-every year.  And whatever you do, don’t marinate in your own stench.

Image & insight courtesy of Scott Adams:  http://dilbert.com/strips/2012-06-10

Why Geekly Certification Matters: The Why, The Journey, and Certification-fu à la Bruce Lee

Back to MSIT People Aspirations:  Here's the outline of my 16-slide deck. I hope you enjoy it, including the clips, the quotes, my slide notes, & the wisdom of Bruce Lee. More importantly, may you find the inspiration for your own journey.  Hai!

1.  The Why

• Competency
• Community
• Career

2. The Journey

• From my conscious decision to become a geek to MCM & beyond

3.  Certification-fu à la Bruce Lee

• Focus on Growth
• Aim Past Your Target
• Apply What You Know
• Make Things Happen
• Attitude

People Aspirations - Why Certification Matters - Jimmy May - 2013-05-02.pptx

Optimizing Your Query Plans with the SQL Server 2014 Cardinality Estimator

One of the gems introduced in SQL Server 2014 is the Cardinality Estimator (CE)—new! improved! & now with more cowbell.  I'm thrilled to be a Technical Reviewer for a superb MSDN white paper authored by my friend, buddy, & pal Joe Sack (b|t). It's exciting & humbling to see my name among such an array of Contributors & Reviewers—including several former colleagues from Azure CAT (formerly SQL CAT b|t).

What’s a CE?

As described on the Cardinality Estimation (SQL Server) page:

Cardinality estimates are a prediction of the number of rows in the query result. The query optimizer uses these estimates to choose a plan for executing the query. The quality of the query plan has a direct impact on improving query performance.

Why a New CE?

The pre-existing CE is more than a decade old.  Both OLTP & DW workloads have changed—& databases are bigger by far than they used to be.  Often, cardinality changes spawned disparate plans (in one prototype, over 78 different plans were generated by the former CE).  Plainly & simply—the CE needed more cowbell.

What’s New?

During SQL14 TAP, SQL Engineer Kate Smith provided a heads up.  Highlights included:

Relaxing Independence Assumption:  The old CE assumed that column values were independent.  Yet columns such as City and State, or Manufacturer, Make, and Model are tightly correlated.  Algorithms in the new CE better account for this.

Join Changes:  Improvements to equijoins, non-equijoins, & join estimates related to primary keys.

Ascending Key Modifications:  Newly inserted data are out of the range in histogram.  The new CE assumes not only that the data actually does exist & also is present at the average frequency of values in the table.  (And the same heuristics also apply to missing values in sample statistics.)

In other words, more cowbell.

Joe provides numerous examples & walk-throughs detailing the behavior of the new CE.

Inside Baseball 

Here’s some behind-the-scenes info.  "Cardinality Estimator" didn't appear in the original title which referred merely to performance tuning.  Who wouldn’t want to read a perf paper from Joe?  Yet the working title belied the true nature of the paper.  The published title provides the precision the topic deserves. 

I won’t reprise the penultimate comma soliloquy that I shared with the editors, but you can learn more here, or pick up a copy of Fowler’s Modern English Usage.

White Paper Metadata

Tile: Optimizing Your Query Plans with the SQL Server 2014 Cardinality Estimator 

URL: http://msdn.microsoft.com/en-us/library/dn673537.aspx

Summary: SQL Server 2014 introduces the first major redesign of the SQL Server Query Optimizer cardinality estimation process since version 7.0.  The goal for the redesign was to improve accuracy, consistency and supportability of key areas within the cardinality estimation process, ultimately affecting average query execution plan quality and associated workload performance.  This paper provides an overview of the primary changes made to the cardinality estimator functionality by the Microsoft query processor team, covering how to enable and disable the new cardinality estimator behavior, and showing how to troubleshoot plan-quality regressions if and when they occur.

Authors: Joseph Sack (SQLskills.com b|t)

Contributers: Yi Fang (Microsoft), Vassilis Papadimos (Microsoft)

Technical Reviewers: Barbara Kess (Microsoft), Jack Li (Microsoft), Jimmy May (Microsoft b|t), Sanjay Mishra (Microsoft), Shep Sheppard (Microsoft), Mike Weiner (Microsoft), Paul White (SQL Kiwi Limited b|t)

I'm confident you'll find the paper as edifying as I did. Enjoy!