Perspective on Continuous Integration at Scale

This blog is by Hrishikesh Karekar (@hrishikarekar) and Vinaya Muralidharan (@vinaya1980) on their talk – Continuous Integration at Scale – at XP Conference India 2016. This post first appeared in Vinaya’s blog site


The word Enterprise conjures up images of big multi-million projects, thousands of people working together, distributed locations, complex products, multiple modules, projects impacting millions of users and mission-critical systems.

All of these are true for our environment.

This is the environment where we have successfully started our Continuous Integration journey and are enjoying great results.

When we embarked upon a journey to modernize and streamline the way we developed software, it was a daunting task. And the change was huge in all aspects – People, Technology, Processes.

But the results are very heartening. From being able to clone environments within minutes, to reducing build times from days to hours to the ability to deliver to customers frequently – we are experiencing the benefits every day today!

The road was not easy. Being a Service Delivery organization, and not Product Development, in itself posed a huge challenge. On top of that we were working with multiple products, multiple technologies and multiple specializations, with integration as our top issue to solve for years. Legacy source control and build systems limited our ability to learn from the industry.

We frequently worked on mega transformation projects for big Telcos. These were always multi-vendor environments demanding heavy upfront planning.

With the multi-product and super-specialization environment, our teams were component teams and not feature teams.

Against this backdrop, we undertook our Continuous Integration journey over 3 years ago.

Technology was our first stop – modernization was kicked off on several fronts – version control, builds, artefact management and test-automation framework. Our new technology stack (including Perforce, Maven, Nexus, Sonar, Jenkins) relied a lot on open source – we learnt from the collective knowledge of the industry!

Process – We supported the technology upgrade with lighter and more agile processes. We cut down on bureaucracy and focused instead on getting to working integrated software quickly and transparently.

People – People took center-stage in this transformation. Developers started being viewed as more than mere resources – we started acknowledging our craftsmen! The sphere of influence and impact of our developers increased and they were ably supported by new tools, processes and technology.

The change was huge and it wasn’t easy. We made a few missteps along the way but we are continuously learning and course-correcting.

Our developers had to learn new ways to code, our approach to metrics had to undergo a re-haul, collaboration between teams took on new significance and middle-management was recruited as an ally!

Many large enterprises may face similar challenges and we would love to share our experience with more people, to assist in our small way, in their transformation.

We have come a long way – however we have much longer to go.

We are already doing several proofs of concept in areas including security, deployment and automation.

The future holds many promises!

Come hear us share our experience at XP Conf India 2016

Why should you send developers to XP Conference India?

XPConference India 2016 is hardly a month away.If you are an engineering manager or director who has developers working in their group, try to answer these questions.

  • Do you know if your developers know how to write Zero Defect Well tested software?
  • Do they know how to write unit tests at all layers of software?
  • Do they work as one team to delivery clean code?
  • Do they know what clean code means?

Screen Shot 2016-07-07 at 4.35.48 PMMore than 60 percent of the cost in a product is not in building it, but in its maintenance. We also have to keep adding new features to an ever growing code base. This problem becomes only worse when you have hundreds of people working on a common code base.  XP Conference focuses mostly on one thing – Clean Code and Extreme Programming Practices.

Clean Code: A Handbook of Agile Software Craftsmanship was a book written by Robert Martin many years ago. Folks like Martin Fowler, Dave Thomas who is the author of the wonderful book Pragmatic Programer , Dr Venkat Subramaniam have been talking about these concepts for decades. However new developers are created every year who come from engineering schools where none of these concepts are taught that well.

This conference focuses on –How to be a professional developer or tester.  

Two days of keyboard driven sessions are broadly into three categories

Listen, See ,Practice

There are three tracks for two days. Totally there are more than a dozen topics to learn from, 9 workshops and a large DevJam session to put everything into practice.

Speakers are all active developers or leaders who actually work daily with developers. One can’t go wrong spending two days with developers who love to code well.

Meet the practitioners announced so far – http://xpconference.in/2016/speakers/

What is my return in this conference?

As a manager or L&D manager, you will benefit by teaching developers skills that they gain by interacting with experts who have been practicing for many years. Each session has been carefully chosen to make sure they are relevant for our attendees and the work they do.

Attendees can learn skills like TDD, Refactoring, Continuous Delivery, DevOps , BDD etc. under one roof. Each of these workshops otherwise would cost hundreds of dollars to attend.

Why am I paying INR 12000/- per person? 

XP Conference is an event for the community and curated by the community. There is not profit making objective behind it nor is it a marketing or sales event. However, we still do incur certain costs to run such an event. While the cost of each workshop itself itself could run into hundreds on dollars or more than INR 20000/-, you get so much more by just helping us cover the costs. If you are not sure ask our last year’s participants who coded in joy for two days.

Reach us at contact@xpconference.in for any question we can answer.

Submissions Are Now Open At XPConference India.

OK It is time. Submissions are now open for XPConference.

SUBMISSIONS CLOSES: Last date for submitting proposals: May 31st (extended to 15th June 2015)

With a theme of – “From Clean Code to Continuous Delivery” – the conference this year will focus on experiential workshops and demo based talks that demonstrate practices.

The sessions must be keyboard driven and must be one of the following two kinds.
1) Something you show to others
2) Something you have others participate with you.

There will be a GIT repo set up for you to pull code from and have others participate with you.

This conference is modeled around XPConference.org the Europe based XP Conference happening for more than a decade.

There are roughly 18 sessions or clinics that we can do. As the topics come along, the selection will happen basis peer reviews.

What should you submit?

1) Any topic that demonstrate clean code or continuous delivery.
2) Things like code smells and design smells
3) End to end product development on the cloud for PAAS
4) Test Driven Development in C for embedded software
5) Branching Strategies that help Continuous Delivery
6) Tools for building continuously for Heroku, Amazon EC2, Cloudfoundry
7) Test Driving Micro Services
8) Tools like Docker for building on the cloud

How should a session be?

The goal of the session is simple:

  • Interactive
  • Enable Concrete learning
  • Create Actionable Outcomes
  • Bring in lots of fun!

Spread the word and participate.

Why Speak?

  • One of its kind in India – While there are already a number of conferences in India that cater to general Agile and Scrum practices, there are few very few specific to Extreme Programming practices which impact everything a developer does.
  • Global stage – XP Conference is a fantastic opportunity to share your technology practice and experience on a truly global stage with over 200 members
  • Foster individual learning and collective advancement with disruptive ways of working and practices that shape the technology of future

International Conference on XP and Clean Coding Practices.

Delivering value: From Clean Code to Continuous Value Delivery
The International Conference on XP Methods, hosted by Volunteers and the community at large , gathers together an international audience of industrial experts and academics on agile and XP software development and is modeled around XP Conference happening in Europe for more than a decade

Mission
The mission of the XP conference in India to further the state of Coding and design and testing for software development by providing a forum at which experts and novices from industry can meet and learn from each other.