Testing

• Leading a Culture of Effective Testing

        This is the best article about the subject I had ever read. Must read not only for testes, but primarily read by all managers. Congratulations and thanks for the article!

• Anomaly Detection, Fault Tolerance and Anticipation Patterns

          John Allspaw discusses fault tolerance, anomaly detection and anticipation patterns helpful to create highly available and resilient systems.  It is a good talk if you are working on alert and monitoring.

• Development at the Speed and Scale of Google

       Ashish Kumar presents how Google manages to keep the source code of all its projects, over 2000, in a single code trunk containing hundreds of millions of code lines, with more than 5,000 developers accessing the same repository.

• The Mindset Change for the Agile Tester

       Janet Gregory discusses changing a tester’s mindset from “How can I break the software?” to “How can I help deliver excellent software?”, using examples and advising how to apply it on agile projects. 

• 7 Deadly Sins of Automated Software Testing

        Adrian Smith covers symptoms, root problems and guidance on recommended solutions for avoiding automated testing mistakes

• Disruptive Testing: The Hunt for Black Swans

          Anne-Marie Charrett advises developing a testing mindset and a tester skillset that helps testers embrace disruption instead of fighting it.

• Programming Should Be More than Coding

        Leslie Lamport makes the case for separating the design details of what a program should do and how it should work from the business of writing code, and discusses how the design process should work.  This is  not a testing talk,  Leslie from Microsoft  was the winner of the 2013 Turing Award for imposing clear, well-defined coherence on the seemingly chaotic behavior of distribute 

• Elisabeth Hendrickson on Building Quality In

      Elisabeth Hendrickson is the Director of Quality Engineering for Cloud Foundry at Pivotal Labs. She is the award winning author of "Explore It!: Reduce Risk and
Increase Confidence with Exploratory Testing" and tweets as @testobsessed.  

• Getting Up  and Running with Selenium for Automated Web Testing

         Emma Armstrong shows how to use Selenium and NUnit to automate web testing for C# applications. The sessions is useful to developers of other languages that Selenium supports – Java, Python, Ruby.

• Automated Test Hell, or There and Back Again

         Wojciech Seliga shares from experience how complex it can be to deal with thousands of tests -unit, functional, integration, performance- for Atlassian JIRA and what they did to bring it under control.

• Performance Testing Crash Course

        Dustin Whittle shares the latest performance testing tools and insights into why your team should add performance testing to an agile development process. Learn how
to evaluate performance and scalability with MultiMechanize, Bees with Machine Guns, and Google PageSpeed.

• Web &  Database Load Testing with JMeter

          Michael Dowden introduces JMeter and explains how to develop a data-driven methodology to determine some of the limits of a web application: max number of concurrent
users, bottlenecks, etc.

• Book Review: "Taste Test, 2nd edition" and Q& A with Matt Jaynes

Configuration management is the foundation that makes modern infrastructure possible.  Tools that enable configuration
management are required in the toolbox of any operations team, and many development teams as well. Although all the tools aim to solve the same basic
set of problems, they adhere to different visions and exhibit different characteristics. The issue is how to choose the tool that best fits each
organization's scenarios.

 GTAC 2014: Presentations

On October 28th and 29th, GTAC 2014, the eighth GTAC (Google Test Automation Conference), was held at the beautiful Google
Kirkland office. The conference was completely packed with presenters and attendees from all over the world (Argentina,
Australia, Canada, China, many European countries, India, Israel, Korea, New Zealand, Puerto Rico, Russia, Taiwan, and many US states), bringing with
them a huge diversity of experiences. Below are a list of talks I watched or planned to watch

• Never Send a Human to do a Machine’s Job: How Facebook uses bots to manage tests

           Roy Williams (Facebook)

        Facebook doesn’t have a test organization, developers own everything from writing their code to testing it to shepherding it into
production. That doesn’t mean we don’t test! The way that we’ve made this scale has been through automating the lifecycle of tests to keep signal high and
noise low. New tests are considered untrusted and flakiness is quickly flushed out of the tree. We’ll be talking about what’s worked and what hasn’t to build
trust in tests.

          Links: Video, Slides

• Opening Keynote -  Move Fast & Don't Break Things

        Ankit Mehta (Google)

       Links: Video, Slides

• Scalable Continuous Integration - Using Open Source

         Vishal Arora (Dropbox)

        Many open source tools are available for continuous integration (CI). Only a few operate well at large scale. And almost none are built to
scale in a distributed environment. Come find out the challenges of implementing CI at scale, and one way to put together open source pieces to
quickly build your own distributed, scalable CI system.

         Links: Video, Slides

• I Don't Test Often ... But  When I Do, I Test in Production

          Gareth Bowles (Netflix)

        Every day, Netflix has more customers consuming more content on an increasing number of client devices. We're also constantly innovating to
improve our customers' experience. Testing in such a rapidly changing environment is a huge challenge, and we've concluded that running tests in our
production environment can often be the most efficient way to validate those changes. This talk will cover three test methods that we use in production:
simulating all kinds of outages with the Simian Army, looking for regressions using canaries, and measuring test effectiveness with code coverage analysis
from production.

         Links: Video, Slides

• Free Tests Are Better Than Free Bananas: Using Data Mining and Machine Learning To Automate Real-Time Production Monitoring

         Celal Ziftci (Google)

There is growing interest in leveraging data mining and machine learning techniques in the analysis, maintenance and testing of software
systems. In this talk, Celal will discuss how we use such techniques to automatically mine system invariants, use those invariants in monitoring our
systems in real-time and alert engineers of any potential production problems within minutes.

The talk will consist of two tools we internally use, and how we combine them to provide real-time production monitoring for engineers almost for free:

1. A  tool that can mine system invariants.
2. A tool that monitors production systems, and uses the first tool to automatically generate part
        of the logic it uses to identify potential problems in real-time.

       Links: Video, Slides

• Building Resilience: How Outages Shaped Etsy's Systems

Avleen Vig presents some of the most unexpected, confusing, hilarious and face-palming events during Etsy's outages to show what can be
learnt from their problems to build more resilient systems.

• How We Learned to Stop Worrying and Start Deploying the Netflix API Service

Sangeeta Narayanan goes over how Netflix got to the current continuous delivery state, the lessons they learnt and the successes they
enjoyed along the way.  

• Scaling Chartbeat from 8 Million Open rowsers to Realtime Analytics and Optimization

Wesley Chow presents Chartbeat's real-time analytics platform and how able to handle the requests in a cost efficient manner using a custom
written analytics engine in C and Lua. He also talks about numerical problems  that arise from scale, and some algorithmic challenges and solutions
encountered while measuring tens of billions of engaged minutes every

• Distribution, Scale and Flexibility with ZeroMQ

Pieter Hintjens explains the basic techniques for building very large distributed systems using fast asynchronous messaging in general, and
ZeroMQ specifically. 

• How Zynga Handles Monitoring at Scale in Its Hybrid zCloud

Matt West explains how to use technologies like CloudStack, Beanstalk, Gearman, mod_gearman, Nagios, nagconf and other tools to monitor
large web applications at scale deployed in the zCloud.

• Decomposing Twitter: Adventures in Service-Oriented Architecture

Jeremy Cloud discusses SOA at Twitter, approaches taken for maintaining high levels of concurrency, and briefly touches on some functional
design patterns used to manage code complexity.

• How Netflix Architects for Survival

Jeremy Edberg discusses how Netflix designs their systems and deployment processes to help the service survive both catastrophic events like
zone and regional outages and less catastrophic events like network latency and random instance death.

• Yo - Notification as the Message Itself

Or Arbel discusses how Yo's platform enables developers and users to communicate using push notifications.

• Facebook’s iOS Architecture

Ari Grant discusses how Facebook is iterating its mobile products, continuing to increase the richness of the content and speed at which it is
delivered. 

• You Won't Believe How the Biggest Sites Build Scalable and Resilient Systems!

The authors discuss about the lessons learned from all the biggest sites on the internet about how to build scalable and resilient architectures.

DevOps and Automation

• Instrumenting Your Business For Success with DevOps

Robert Benefield offers a pragmatic overview for discovering operational indicators that provide valuable insight in running and improving
online services.

• InfoQ eMag: DevOps War Stories

In this eMag we share real war stories from organizations that have adapted and evolved towards their own DevOps culture. The successes and
the failures. The breakthroughs and the steps backwards. The technical advancements and the people changes.

• Bringing Windows into Submission with Puppet and PowerShell

Paul Stack discusses using PowerShell and Puppet to administer Windows machines, showing how to configure a Windows server and set up a
development environment in short time.

• Book Review: Test-Driven Infrastructure wth Chef - Second Edition

In the first chapter of the book "Test-Driven Infrastructure with Chef, 2nd Edition",Stephen Nelson-Smith discusses the philosophy behind Test-Driven
Infrastructure. He lists two fundamental philosophical points:

1. Infrastructure can and should be treated as code
2. Infrastructure developers should adhere to the same principles of professionalism as other software developers.

Then, he dives deeper into each of them….

• InfoQ eMag: Cloud Automation and Management

In this eMag, we curated a series of articles that look at automation in the cloud and management at scale. We spoke with leading
practitioners who have practical, hands-on experience building efficient scalable solutions that run successfully in the cloud. 

• Jeff Lindsay on Best Practices for Working with Contairners and Docker

Jeff Lindsay is the author of Dokku and many Docker related OSS projects. He is the co-founder and principal of Glider Labs, a devops
consultancy specializing in modern, Docker-oriented system architectures. He was involved in the early stages of Docker development, and has worked with
many other organizations such as Twilio, Digital Ocean, NASA Ames on distributed systems and developer platforms.a

• The Evolution of Continuous Delivery at Scale @ Linkedin

Jason Toy talks about the evolution and history of LinkedIn's release strategy. Jason
Toy drives the direction for build automation at LinkedIn, focusing on the
commit to release pipeline with the ultimate goal of allowing developers to
move code from dev to prod in 30 minutes.

• Continuous Delivery: Building Systems that Can Pivot

Sam Newman, Zhamak Dehghani make an introduction to microservice architectures,
explaining why they are ideal for organizations that want to optimize for fast
speed of change.

• The Art of  the Builds

Hans Dockter presents an in-depth treatment of build systems concepts. Hans Dockter
is the founder of Gradle and Gradleware. He is a thought leader in the field of
project automation and has successfully been in charge of numerous large-scale
enterprise builds

• Software Development & Architecture @ LinkedIn

Sid Anand discusses the architectural and development practices adopted by LinkedIn
as a continuous growing company.

• The Facebook  Release Process

Chuck Rossi unveils some of the tools and processes used by Facebook for pushing new
updates every day.

• Deployed in  60 Minutes: Increasing Production Deployments from Six Months to Every Hour

Sponsored by Twilio. Matt Makai explores why deployments are difficult and shows
solutions with case studies on how other organizations cut their production
deployment times down from months to hours.

• Continuous Delivery for the Rest of Us

Lisa Van Gelder provides simple tips and tricks for improving delivery without
investing lots of time up front creating complex deployment frameworks.

• Continuous  Deployment the Octopus Way

Jimmy Bogard takes a look at how Octopus enables continuous delivery and what it
offers over standard tooling.  Jimmy Bogard has delivered solutions ranging from shrink–wrapped products to
enterprise e–commerce applications for Fortune 100 customers. Jimmy is a member
of the ASPInsiders group, the C# Insiders group, and received the MVP award for
ASP.NET in 2009-2013. Jimmy is also the creator and lead developer of the
popular OSS library AutoMapper.

• From Code to  Monkeys: Continuous Delivery at Netflix

Dianne Marsh presents the open source tools used by Netflix to keep the continuous
delivery wheels spinning. Dianne Marsh is a Director of Engineering for Netflix in Los Gatos, CA, where she
leads a team responsible for tools and systems used by nearly all engineers in
the company for continuous integration, delivery and deployment to the AWS
cloud.

• Development, Deployment & Collaboration at Etsy

Daniel Schauenberg provides insights into how Etsy develops software and what tools
and processes they utilize to help them achieve their goals.

• True Story  of Continuous Happiness by Continuous Delivery

Sergejus Barinovas discusses the challenges, failures and actual steps required to move
an organization to continuous delivery and as follows - continuous happiness.

You may wonder what I am doing now?  I am a developer in our telemetry team and I

• report to one manager, but spend half of times to help other manager to develop dev ops tools since who you report to is not import, but what value you can add does matter.
• full time seat with my customer (SQL Azure developer) and solve their issue in matter of hours (not days or weeks).
• treat my customer's request as highest priority, saving their time is more important than saving my time.
• write an automatic bot to resolve SQL Azure Livesite issue automatically with less than one minute resolving time, and save our Livesite engineers hundreds of time (they don't get wake up during night).
• organize Yamjam for our service to bring the bridge between we and customer.
• help to host other team's site which is used by many time when their team does not exist any more.
• try to use the principle I learned to my project every day, such as unit testing,  continuous integration, A/B testing, deployment automation.

"Sharing the responsibility and allowing developers to contribute to supporting the systems they create is the first step towards reaping the benefits of effective testing."

"When testing is given to another team, it's almost always manual and when it is automated it's usually an end to end test that is difficult to maintain. Isolated testers rarely have the insight nor capability to test on a lower level within the application and especially not on a unit level where the majority of effective verification is done!"

"The most effective testing I've been involved with was for projects where the outcome of the software was the foundation of everyone's focus. Instead of being told what to do, for example what report to create, I knew what business outcome was desired and could help decide what reports were necessary."

"Testing without purpose is like a form of punishment! Tests that don't help actually hurt, they add to the code that has to be maintained. Furthermore, useless tests lead to confusion in the long term as tests are highly regarded as sacred descriptions of how the system should work."

What do you think?

English  translated
version: http://www.microsofttranslator.com/BV.aspx?ref=IE8Activity&a=http%3A%2F%2Fwww.infoq.com%2Fcn%2Farticles%2Fdouban-dev-management

Based with Douban’
Engineering Vice President Duan Nian Wei Zhuang table conversation rather than
into, the two sides mainly discussed how to give the team a set of rules, how
to transmit values, right motivation strategies, how to examine topics such as
engineers.

Introduction of guests

Duan Nian, incumbent Douban
Vice President of engineering. Started programming (BASIC) in 80s, individuals
interested in entering the software industry with the support of, successively
serving in companies such as Huawei, new technology, joined Google in China. In
software development, testing, software, team management and other fields are
involved, recently focused on team-building, engineering culture. Interested in
ways to improve team productivity and personal skills.

Profile

Douban’ s development
management is based on a set of constraints and rules. Constraints are
fundamental, rules based on constraints to development. We have three basic
constraints:

1.Final evaluation
criteria depends on the contribution to product line

2.To do things the right
way

3.Must have technical
objectives

1st is arguably performance
recognized principles, that is, what kind of performance can be recognized.
writing 1000 lines of code is not a performance indicate, what value you
bringing is. When you don't have a direct contribution to the business, but increasing
productivity, this is also count.

2nd is that we do not accept
low-quality code. Good taste of engineers natural with it.

3rd can also say we are
asking members of the pursuit of code. If we are looking for a person, if he
only work as a task, lack of enthusiasm for upgrading their technology itself,
so even if he was again well, we refuse. Such a person might just focus on
performance, rather than focus on smarter, more efficient way of doing things.

Based on this three-point
constraints, we have formulated a number of rules, these rules can lead to
other rules, but each team also produce their own set of rules.

For example, each code must
to be do code review, this is a general rule. This rule needs the support of
tools, which is why we develop CODE platform in the first place. Our code
review is a process of relative autonomy, not from top to bottom or from bottom
to top. Basically, every team will be formed inside one or two people with code
cleanliness, and they're finding low-quality code for such a role.

Code review tasks, under the
CODE platform, derives from other rules, such as in the merge code before you
implement CI, as well as branching rules, submitted to the rules of the code,
and so on. Of course, this also requires tools support. CODE as a platform can
be implemented well to accommodate these rules.

Within each team, if your PM
want to decide how to do distributing the task, they may have different rules.
Some teams had created 20% not function development rules, specifically do with
20% time to generate long-term value development, such as debt-repair
technique. Some teams take consultative manner to distribute workloads that are
not visible at a time. These are derived from the team's pursuit of technology.

Overall, the constraints are
the same, the rules can be adjusted.

how to formulate the rules

The management 3.0, this book
says: good managers is not the rules of the game makers. We tend to let the
team his own team rules in a democratic way, I try not to meddle. Of course, in
the early stage of team development, and you can also try to set some rules,
but you will find that these rules will be replaced by new rules arising from
within the team.

As managers, we could not
help but to like a game designer to design rules, but this is impossible to
define the rule, we should not do so. We ought to emphasize constraint defines
the boundary rules to ensure no conflict rules with constraints.

Some are more attention to
consistency, especially ideological consistency, unity of values, I disagree on
this point. I think the unity of this thing makes no sense. The so-called
consensus, there are generally three levels:

1.Vision: Is "what to do
and what not to do." For example to put ads on the page, it should do it,
everyone will have their own opinions.

2.Value:  Is the
"how to do things," under the vision, reflected by rules and
constraints. I think values is not something pasted on the wall--the mantra on
the wall, are generally less stuff.

3.Rules and restrictions.
That is in practice. Rule is a thing which can easily be copied, such as the
development process, you see everyone with a pull request to submit, you can
easily do so. In this process, the values have been enhanced.

For me, I prefer all
consistent in behavior rather than to pursue the thought consistent. Rule
creation process should be democratic, the process requires conflict,
collision, compromises--because our thought is not consistent and rule once it
is established, every individual should abide by the rules.

How to encourage
cross-team collaboration and sharing

In early time, our company
only have no more than 20 people, was not yet divided into product lines, all
the tasks in a pool, open on the Trac, if you interest one taks, do it by
yourself. Of course, there are some constraints, such as a practice is
"maintains its own code." After year 2009 , in order to solving the
ownership and rapid response to keep small teams, we entered into a product
line approach. In the line mode, engineer's job is relatively fixed, not like I
used to walk a public pool.

This horizontal linkages
weakened between engineers, good experience, systems, principles cannot be
shared across product lines, while engineers also are limited to a product line
within a long time may seem boring.

So last year, we used a lot
of energy to push cross-team collaboration. First approach is to establish a
common pool, giving individuals more chance of the outcome, but there is no
particularly good results. Now we focus on the performance rules, makes
cross-team's contribution can be recognized by bean performance system,
although they can't say a lot, but compared to last year's attempt is more
suitable for some.

Incentive mechanism

Our use of incentives very
carefully. On one hand you have to say they saw the employee contribution, care
about the contribution, on the other hand you have incentives to avoid
excessive and cause deterioration, becoming inspired to do spontaneous behavior
is one thing.

Last year, we have an
employee voluntarily cleaning up unwanted data in a database, put a lot of
spare time, let the database access speed has been greatly enhanced. So, I
should give him a bonus? Obviously, this is a very valuable and should be
encouraged to contribute, but instant cash rewards are not the best way.
Because the direct cash incentives would likely lead to motivation to change.
And sharing is, we also considered to share a points system, but we're very
concerned that would metamorphose into an internal drive to an external drive –
not the reward, we will not share it? And you set the score, some tasks more
credits, and some tasks less integral, and would lead to "pick"
problem.

Incentive balance how can
this thing do? I agree that bonuses and salaries had better go with the
performance appraisal, rather than an event-specific bonuses – would lead to
spontaneous behavior modification is on the one hand, on the other hand is very
likely to be fair. For employees doing good rates incentive is as it should be,
but not necessarily with money. Coat of arms of the CODE system is a good
instant incentive--the coat of arms of saw his effort on my behalf, can also be
displayed for others to see, you can have very good communication without
adverse interference on the internal drive.

Assessment systems mainly to
solve how to evaluate engineering problems. Basically, we did not set specific
quantitative targets, primarily two basic points: one is face to face, face to
face with tech lead assessment of every engineer. Another is public, is that
all of the engineer's assessment basis is open to the public. We did an
assessment every six months, and 3 days each time 5 or 6 tech lead, we can
discuss and even PK, all kinds of reasons to have an article. Now all of the
assessments I have participated throughout the development team now, more than
130 people, basically I need everyone, long-term growth in the future, I hope
that the assessment process into the Commission's Forms.

However, I think the
assessment of performance evaluation is not the most important part. Evaluation
the most important part of goal-setting and should be checked on a regular
basis, there has been more, such as how to delegate and how to target
selection, and so on. Administrators manage objects not people, but the system:
for the team this system, how do you make team recognized objectives and
constraints, how to make the team are capable of forming their own rules, to
harmonize these rules with the goal, this is the team managers should focus on
things. For people management, managers should do a maximum incentive level,
people further down the assigned things don't fit anymore. Team themselves as
long as there is a goal, it will spontaneously go, you don't need to pay
attention to the team what to do.

Now market Shang many
management of theory, are has respective stressed of points, like KPI or bonus,
actually you will found many theory Zhijian is conflict of, because they no put
company overall into considerations, but only see a a level, a part, on with
this a part derivative out set management theory, looks is has truth, but
really to practice up, which many are just is "looks is beauty" just.

Finally recommends two books:
a book is mentioned by me earlier management 3.0, which provides a very good
framework and managers need to consider the dimension. Although the author of
this book in order to snap "management in complex theory" this
"subversive" theme, referring to a lot of classical management theory
also exist in concept, deliberately using different names or description--from
this point that the author is a little "trick", but the book is still
a good book. The 3.0 refers to the complexity of the theory of knowledge
management can be found in Melanie's book, the complex, the complex, this book
introduces the basic theory of many complex areas, such as cellular automata,
ants, chaos theory, non-Turing machine, bio-information engineering and for
understanding complex systems has been a big help. Of course, you want to know
more about complex theories, Hofstadter book is not to be missed.