In the early days of the cloud, most people thought of it as a way to host applications: it was an IaaS (Infrastructure as a Service) world. I liked to call IaaS the "xcopy" model since it essentially means you take a VM and copy it into the cloud (with some network configuration, of course). It's a fairly simplistic model but, obviously, of great utility since the cost savings for an enterprise IT department can be quite significant. With large numbers of applications migrated as IaaS workloads IT departments can reduce or eliminate their dependence on their own data centers with consequent savings in real estate, hardware, electricity, air conditioning, and the like. It's a great story and it's probably still the predominant model of cloud computing today.

But in many ways the IaaS model is only the first step of cloud usage. Let me give you an illustrative example of what I mean. I'll use an IoT scenario but it's evocative, hopefully, of many other types of cloud applications. In this case the customer had deployed an application that consisted of a VM running in the cloud, some devices, and some logging, which was then analyzed and the results displayed on a custom dashboard. Sort of like this (please forgive my terrible drawing skills!):

That's all well and good. The application worked and was demonstratable.

But then I asked some questions.

• What happens when you're really successful and you're monitoring perhaps thousands or tens of thousands of devices?
• How do you detect patterns of usage? Patterns of failures?
• How can you predict component failures or do predictive maintenance?
• How can you watch for potential security breaches, or fake traffic?
• In short, how can you write all the software you need to do this, quickly and reliably?

With many of the new services available today in Microsoft Azure, I suggested, you can do all of these things, and a lot more, today, with very little coding. Check out the diagram below:

So the devices send events (in REST or AMQP) which flow into Azure Event Hubs, which today process in aggregate billions (literally) of events per day. Azure Event Hubs provide a scalable way to do massive event ingestion; you only pay, however, for the bandwidth you need and use. In this case the events are picked up by Azure Stream Analytics which can do intelligent filtering and correlation. The data can then be placed in Azure Storage of one form or another which Big Data technologies such as Azure HDInsight (Hadoop) can analyze it and where data scientists using Azure Machine Learning can build experiments, train models, and quickly build applications to do predictions. PowerBI provides a visually pleasing user interface -- a dashboard --  which can be directly interfaced to Stream Analytics or a number of data sources including Azure SQL. Azure Service Bus provides a simple way to send queued or publish-and-subscribe messages and commands back to the devices. If you like, small little objects -- actors -- using Project Orleans today, the microservice-based Microsoft Service Fabric tomorrow, can implement server side logic for each individual device. And there are lots of other components that can be used to build even richer applications. And of course with the Azure Portal and Azure Application Insights your DevOps can monitor and manage your application.

Here's the key message: all this required very little code. Rather, the application -- and so many like it in Microsoft Azure -- can be wired together quickly. Comprised of services readily available in Azure, all fully tested in a wide variety of use cases and configurations, such applications reduce the need for to write custom code. (Prefer the open source versions -- say, Apache Storm in place of Stream Analytics? No problem!).

To me this sort of architecture represents the real value of cloud computing, that is, the ability to quickly wire together these high-value components with a minimum of coding (in some cases, none at all). So many IT professionals I talk to think of the cloud as purely a hosting opportunity but in fact the services available make massively scalable, feature rich applications easily accessible to enterprise IT.

Building a breakthrough application? I'd love to hear about it!

Want to see what I mean by that? Check out the case studies we're posting on our team blog, located here.

Happy reading!

Many have asked for the slides so I've attached the deck to this blog post.

Enjoy, and happy architecting!

Enterprise Architecture in the Time of the Cloud.pptx

Here’s a simple example of what I mean by enhancing innovation: one of our more straightforward standards in Microsoft IT was that all IT applications had to use Active Directory for authentication. Some of the benefits from this (utterly non-controversial) standard were:

• No one wasted any time or resources building their own authentication mechanism; instead their focus was on new features;
  
  
• Third party applications we purchased had to comply with this standard;
  
  
• And therefore our operations staff only had one identity mechanism to manage.  

Thus standards reduce operations cost while focusing development on those areas most likely to provide business value (and not reinventing wheels that were perfectly suitable and in fact where considerable investment had already been made).

Standards for integration mean that applications can interoperate and new applications can smoothly join the ecosystem; standards for development mean that applications use consistent methodologies and libraries so they function predictably; standards for data mean that applications can understand each other’s models and thus take better take advantage of IT’s core asset, information; standards for security lead to a demonstrably and quantitatively more secure environment.

One of our more interesting standards was one that required applications to “stay current” on latest versions of system software (OS, database, and so forth). We created this standard in response to an internal study that clearly showed dramatic decreases in support costs (number of tickets, average time to resolution) for applications on n or n-1 versions of system software.

So how do standards get created and operationalized? While standards were the responsibility of the Enterprise Architecture team, EA only managed the process. A cross-group team called the Standards Review Board, composed of senior individuals, reviewed all proposed standards. Here, in brief, was the process we followed:

• Anyone could propose a standard;
  
  
• Proposals were reviewed by the SRB and if reasonable, a working group of SME’s was created to hammer out the details. We tried hard to get diverse opinions on the WG so that no standard would reflect parochial interests of a single group;
  
  
• The final version was review by the SRB. The final version had to include:
  • The motivation for the standard;
  • The final text;
  • An impact statement
  • An adoption plan, including, if needed, how to
    retrofit existing applications

Over time, we learned a lot about how to streamline the process and how to drive change management through the organization. For example, we chose specific technical areas on a quarter-by-quarter basis where we wanted to prioritize; we scorecarded applications during the Architecture Review on improved compliance to standards; we assigned a PM to track the working group activities and ensure they met their deadlines.  

We said that “standards focus innovation;” and while standards require a certain amount of investment (and executive buy-in) their return is significant.

• Be clear on your purpose. We used the ARB to examine projects from several perspectives: a technical overview, including key technology choices, context diagrams, and data models and flows; adherence to standards; and being a healthy, "good citizen" in the overall ecosystem of applications and data. We specifically and intentionally avoided certain areas like headcount allocation and project costing as these were covered in other governance forums.  
• Pick your projects. An architectural review to be meaningful and relevant takes time and therefore you should focus on those projects where there is the most investment and where there is most risk – and any other criteria you think important. (For example, I reserved a few sessions for projects I deemed “strategic” – projects in very early stage but over time would be transformational – such as master data projects.) Don’t try to do everything. Our ARB met once a month for a half day and reviewed 2-3 projects at most.
• Of those, only review major releases, for similar reasons.
• We adopted the idea of an “engagement architect” who would work with project teams to prepare for the ARB. The “engagement architect” met with the project team several weeks in advance and ensured that the presentation was of high quality, was accurate (more on that later), and covered all the points that the Board wanted.
• Do the ARB early in the project lifecycle. Prior to my tenure, early ARB's were performed late in the development cycle and merely ended up as “red flag” exercises which resulted in time-consuming rework and delayed projects -- and needless to say, were therefore not held in high regard. If you review early, you have an opportunity to make mid-course corrections and keep the project on track. There is no value in doing a review late – except as noted in the next bullet.
• Optionally, you can do a checkin late in the lifecycle – to make sure suggested changes made it in.
• One of the innovations we made was to scorecard apps based on overall improvement in each of the reviewed areas. If for example the new version of the app was “better” from a data quality standpoint, we gave it a +1. If no changes were made in the data quality area, it got a zero. If it regressed in our view, it got a -1. (In many cases it was not a project goal to improve in certain areas to getting a "0" was not necessarily a bad thing.) It There were 8 such areas, and we could then provide to the CIO an at-a-glance view into how well the ecosystem overall was improving, or not.
• The ARB must have the power to stop a project. This gives "teeth" to the ARB function. In our case we exercised this power very, very rarely -- in fact, only once, where we asked a project team to align with another having a similar goal.

One of the great benefits of the ARB, in my opinion, was that it gave the solution architects a chance to present in front of their peers. This led to great discussion about technologies and technology tradeoffs, it educated the broader architecture community about what was going on, about what approaches were being taken, about assets they could take advantage of, and overall led to a sense of community. In many cases as well it was a great forcing function for the project to be explicit about the technical decisions they were making.

There’s no question that this is true. The PC of course was such a disruptive change; as was the internet, big data, smartphones and so on. It almost seems like one day there weren’t there; and the next, they were; and the world changed as a consequence.

Yet our natural instincts day to day are to do something very different. In so many efforts we do version 2.1, then 2.2, then 2.21, and so on, while somewhere, someone completely changes the game forever.

In my career I’ve been lucky enough to be part of some of these disruptive changes – I’ll call them “breakthroughs,”  because they really do remove barriers and break us through into new places. Let me tell you about them.

We often think of breakthroughs as reserved for the ultra-smart, and of them only those who take the big risks, spend the big bucks, and so on.

I don’t think so.

To me creating a breakthrough culture is all about a mindset. In fact, in my experience, a breakthrough culture is not any more expensive, time-consuming or costly than one of incremental progress. Check this graphic out:

If you look at the chart, you’ll see a breakthrough requires about the same amount of effort as stepwise progress. This has been my experience, having lived through both "modalities."

So why not choose a breakthrough?

Before you answer know that, of course, there is a difference. Breakthroughs lack what we humans crave, namely, predictability and thus the ability to be controlled. When you’re doing (say) Release 2.21a of some product, you usually have a very clear and precise idea at the beginning of what will be in it, how well it will work, how much it will cost, how many resources, blah, blah, blah.

Breakthroughs – well, you don’t have that level of certainty, and for those folks who need it, that’s a very scary proposition indeed.

So how do you build a mindset for breakthroughs?

I don't have all the answers but perhaps this will help. My old friend and coach Steve Rosenthal of Gap International once said something that changed my entire outlook on life. “Conversations,” he told me, “change the world.” Then he paused for effect. “What else does?”

It was like a lightning bolt struck. What else indeed?

Think about it: it’s true, isn’t it! Innovation is a remarkably social activity when you think about it.

So, if conversations change the world, and breakthroughs are as cheap as incremental improvements, why not have conversations for breakthroughs?

I’ll have more to say about this in a future post.

In September we in the LM Architect community enjoyed a superb keynote presentation from Microsoft’s Patrick Chanezon at our annual Architect’s Workshop, which was attended by over 200 people in person and another 200+ virtually.  In addition to a great overview of current cloud computing technologies, Patrick stressed the importance of Architects maintaining their technical edge, and he asked what we do in LM to help Architects do just that.

I couldn’t agree more, as clearly the most important person on any program is the one leading the team building your customer’s solution!

To meet our ongoing need for trained technical leaders, the LM has an Architecture Development and Qualification Program (LM ADQP) that originated in 1992.  The objective of the ADQP is to build a pipeline of technical talent who can design and develop complex technical solutions as well as lead the technical teams on (then IBM and now) Lockheed Martin’s programs. 

The ADQP focuses on identifying aspiring engineers with technical and leadership potential, then growing them through a prescribed career path.  There are four main parts, including formal education, skills development, mentor and mentee relationships, and on-the-job coaching while the architect is inleadership roles on programs of increasing scope and complexity. 

My mantra is “Identify, Grow, and Recognize” our future and current technical leaders!

Architects who have achieved the education and experience standards established by the ADQP program and demonstrated that ability through sustained, successful leadership are eligible for recognition by one of three titles:

1. Qualified Architect, having successfully led a small program of defined size and complexity
2. Advanced Architect, having successfully led at least one large program of defined size and complexity
3. Master Architect (the highest level), having successfully led multiple large and very large, complex programs.

There are currently about 500 credentialed architects in Lockheed Martin who have earned this unique credential.  They are called upon to lead Lockheed Martin’s most important proposals and programs and are a key factor for ensuring consistent technical execution across the corporation. We believe the ADQP helps LM deliver technical solutions with lower risk, higher quality, and excellent overall performance.

Dr. Poulin can be reached at jeffrey.poulin@lmco.com.

About Lockheed Martin:

Lockheed Martin products range from military aircraft, missiles, space vehicles, Unmanned Aerial Systems (UAS),submarines, and Littoral Combat Ships (LCS).

Headquartered in Bethesda, Maryland, Lockheed Martin is a global security and aerospace company that employs approximately 113,000 people worldwide and is principally engaged in the research, design, development, manufacture, integration and sustainment of advanced technology systems, products and services. The Corporation’s net sales for 2013 were $45.4 billion.

© 2014 Lockheed Martin Corporation. All Rights Reserved.

Here in Microsoft’s TED (Technical Evangelism and Development) organization, we believe in hands-on architecture. A good part of our charter involves working with customers on amazing new applications that take our platform in new directions or demonstrate exciting new use cases. Here are some examples of “epic”
technologies and applications we’re working on now:

• Our Nitrogen.io Internet of Things framework, originally developed for Raspberry Pi and now expanded to a wide variety of devices and protocols, is an easy way to securely connect devices using JavaScript and node.js. A number of the startups in our IOT accelerator (co-sponsored with American Family Insurance) are now using it to connect their gadgets to the internet. Check it out!
  
  
• Several of our architects and developers hosted a small, construction-focused information provider for a hackfest in Microsoft’s Building 20, specially remodeled for these sorts of events. The actual data in this case had to remain on-premise for contractual reasons, but the search index (using SOLR) was hosted in the cloud, and  GPS-enabled, location-aware mobile client running on Azure’s Mobile Services (for iOS, Android and Windows Phone) was built. A cool example of a hybrid app!
  
  
• As John notes in his blog, we hosted a number of companies for a hackfest around the actor-based framework called Project Orleans. While Orleans was originally created for gaming, we worked with, among others, a financial services company to help them with an innovative Azure and Orleans-based portfolio modeling application. The actor paradigm provides an easy on-ramp on to the cloud, a stateful middle tier, and transparent scalability – all of which make it attractive for large-scale cloud applications. We’re also seeing a lot of application for actor models in Internet of Things apps, where an actor can be a virtual reflection of a device, or group of devices.
  
  
• As part of the Duke of York’s iDEA (Inspiring Digital Enterprise Award) program, we have been helping to develop a cloud-based STEM curriculum, with code and applications, for UK schools.
  
  
• With a number of companies we are using new capabilities in Azure such as HDInsight and the exciting new machine learning workbench AzureML to unlock new insights in data. In particular, it’s now possible not only to capture but to analyze the reams of data produced by device telemetry to build, for example, applications that can predict the failure of a given part and schedule maintenance before the failure occurs.

We’re keeping busy, as you see! – And we have a lot more in progress, so stay tuned for more.

1. Build for the future. This seems obvious, but in practice it’s commonplace to trade off a good design for something that “just works.” The result? I’ve seen any number of v1.0 projects that have to be completely rewritten for v2.0, and I bet you have too.
   
   On the other hand I recall back in the early days of Lotus Notes that Ray Ozzie and his team put a strong emphasis on a durable architecture.  They wanted an application written against the Notes v1.0 API to run without change in v2.0, v3.0 and so on, and while many, many features were added over the years, the fundamental architecture, the foundation of the application remained essentially intact (disclosure: I worked on Notes for a number of years).
   
   Do it right the first time; you’ll reap the benefits for a long time to come.
   
   
2. Fix the obvious bugs first. How many times have you been faced with a bug that continually eludes your every attempt to squash it? And – I’ll bet – because you’re great engineers and love hard problems over the easy ones any day, you say, “I’ll fix this null pointer” or whatever as soon as you’re done with the Hard Problem.
   
   Fix the null pointer issue now. As often as not this will actually fix the Hard Problem, and, once it does, the root cause and consequences of the Hard Problem will suddenly become obvious. Trust me on this one; if it sounds like I’ve learned this one the hard way, it’s because I have.
   
   
3. Release early, release often. I know this is a core tenet of agile engineering, but it surprises me how often we ignore this, and instead engage in a year-long (or more) death march to build something that somehow misses the mark. It’s always better to be in-market with something that’s seventy percent of what your vision wants it to be than to be waiting for Ultimate Perfection in Code to be finished. This applies not only to products you release externally but also – if not more – to IT projects.
   
   When you’re in market, customers tell you what’s important … and they know better than you.  
   
   
4. Talk about it with somebody. Isn’t it amazing how many times you’re faced with a hard technical challenge, then go over to the office next door and whiteboard it with a colleague? And suddenly – even if your colleague has precisely nothing to contribute other than a willing ear – it’s all very clear to you?
   
   Even better, talk about it with someone who is not on your project team. As Einstein once famously said, “if you can’t explain it simply, you probably don’t understand it,” and maybe that’s the key: explaining the problem out loud forces you to actually systematically understand it.
   
   
5. There’s no ROI in refactoring. OK, I get it, sometimes you just have to refactor the code. But remember that this “elegance” is not in and of itself adding any value. It’s easy to waste tons of time moving this routine over to that namespace, to change the API’s to make them more consistent, blah, blah. And there may be real value there but the elegance of the code is not part of that. Never forget that you are saying, in effect, that all this infrastructural work is more important than time to market – and it may be, but know that you are making that tradeoff.
   
   What will add value are the features that may be unlocked or enabled – that is where you get ROI. Moral: don’t go crazy.
   
   
6. Breadth and depth. This is more about your own development as an architect, but it’s useful. Here at Microsoft in career discussions we often talk about the “T” model: be technically very deep in a few areas, but have broad awareness of the technical landscape as well. You can’t be successful in the long term without both: you may be an expert in (say) SQL technologies, but without knowing about the capabilities of Hadoop and other noSQL databases – even if you’re not a master of them – you run the risk of not taking advantage of the best technology for the job.  

Well, that’s it for this post. I do have more – and will post them from time to time – but I’m interested in hearing yours!

One of the topics that frequently comes up in briefings and in discussions with customers is the evolution of roles in enterprise IT as we enter the cloud era. Historically, IT departments have organized in the following way: relationship managers (often called account managers or solution delivery managers) connect with their business counterparts and help translate business goals into technical deliverables, with the most prominent artifact typically being a requirements document for a given application or application ecosystem (my frequently used term for a group of cooperating applications that create an overall business solution). As many of you know, creating a clear, lucid, actionable requirements document is actually quite a high art, and those that can do it are, and should be, highly valued.

Implementing the requirements documents are technical teams, using comprising developers, testers, UI specialists, and database analysts.These are the people that create the applications, usually following an SDLC of one sort or another (which could be more or less waterfall, agile, and so on).

Once the application has completed unit, feature, and user acceptance testing it is deployed and (again we are talking traditional environments) it becomes the responsibility of an operations staff to keep it running, ensure it is backed up, provide the right set of hardware network resources to meet its SLA, and so on and so on.

There are a lot of handoffs here, as you can see, and these introduce unwanted latency into IT.

It's now mostly incontrovertible that the cloud changes IT roles. My experience is that there are two fundamental, and equally essential strategies managers must address in this new world: that for migrating existing applications to the cloud, and second, for becoming cloud-native. Both of these require building new (or strengthening existing) muscles. Why? Because above all, using the cloud  means you can run your business much, much faster and more efficiently. And that means your teams must be able to run at cloud speed!

First of all, in my view, the cloud era reinforces the need for the architect function. It’s critical that technical investments provide maximum business value, and architects, who understand both business process and technology, are uniquely suited to provide leadership and guidance on how to best exploit the advantages the cloud provides for their businesses.Enterprise architects optimize ecosystems of technologies to meet business needs; business architects (to some degree superceding relationship managers) connect business to application functionalities; information architects manage that most precious of enterprise assets, data, using canonical models and techniques like Big Data and machine learning to extract new insights.

Second, the user experience takes on new importance the simplicity – indeed, the enjoyment – of an application is very much tied to its usage and success. In the old days it was about reducing the number of gestures to accomplish a task; while that’s still important, now the field of scenario engineering looks to make an entire workflow end to end as seamless and as straightforward as possible whether it be for one user or a group of people collaborating.

And, third, the most obvious evolution in some ways is represented in the term “DevOps” which brings together the previously siloed functions of development, test and operations in an agile way. This very healthy consolidation of skillsets into project teams means a far healthier collaboration between previously separated and isolated teams. The successful deployment and operation of a service (no longer limited to a single application) becomes now a shared responsibility and accountability between developer (who builds in the right kinds of monitoring and self-healing, in addition to business logic), and operations (who among other things creates automated scripts to use the new tooling and provide the right sorts of responses) – with counsel and often leadership from architects. DevOps, history will show, makes IT overall far more responsive. (Check out the excellent DevOps blog and Volker Will’s great Technet blog!)

The objective, of course, is to massively streamline the business idea-to-deployment cycle. By eliminating the siloes and by creating agile, collaborative teams combining areas of expertise and by taking advantage of the on-demand capabilities of the cloud: you can do this!

I’ve put together a slide (above) on the evolution of IT skills and have shown it in a number of customer briefings and group presentations here at Microsoft. It’s just one point of view, however, and I’d be very interested in your comments.

Best to all,

Barry