Big Code, Batch Changes

Automating large-scale code changes across repositories may allow Data Engineers and Developers to work up to 80% faster. Can this address the ‘Big Code’ problem?

Given that big data is set to be one of the leading technological innovations of the incoming decade, it is almost an indispensable necessity to solve some of the most important problems pertaining to it: primarily, the ‘Big Code’ problem.

The Big Code Problem

While the ‘Big Code’ problem has several facets, it generally refers to the ever-increasing complexity and size and the codebases that organisations have to maintain. In fact, even small firms with basic software needs “may have to maintain tens of thousands of lines of code that is written in multiple languages, uses widely varying data structures, incorporates dependencies from dozens of upstream projects, and so on.”

In order to handle the aforementioned, most firms usually resort to simplistic tactics such as, (i) eliminating lines of unused code; (ii) using microservices for specific sections; (iii) expanding the use of comments for retraining and troubleshooting or (iv) using specific frameworks (instead of personal preferences) for various architectures. While all of these have proven to be mildly effective over the past few years, the problem of retraining massive sections of big code with high degrees of complexity is as persistent as ever. In fact, recent research from US-based Sourcegraph has found 94% of their survey respondents to suffer from big code problems, with over half reporting they had over 100 times the code today compared to what they had ten years ago.

According to US software magazine SD Times: “As a result, the report found development teams are finding it harder to include time for new hires to be productive, understand code dependencies, manage changes, provide visibility, and have effective code reviews. Additionally, 74% of respondents reported that their teams avoid updating code because they are afraid of it breaking dependencies.” Given such a scenario, it is almost necessary today to ensure retraining times are cut down and big code problems are met head-on.

Batch Changes for all?

Founded in 2013 in San Francisco, Sourcegraph Inc. is one of the world’s leading web-based code search and navigation tools, being used by developer teams all over the world, including DevOPS teams in firms such as Amazon, PayPal and Uber. It is set to be the primary mainstay for enterprise coding for years to come – especially given the new features that promise to cut time by almost 80%, compared to older methods.

In an update primarily aimed at accelerating ‘development velocity’, i.e. the metric in agile development used to represent the amount of time work teams handle during a sprint, Sourcegraph has now developed for features that will allow firms to “automate and track large-scale code changes across all their repositories and code hosts, saving up to 80% of their time versus existing manual methods.” This is expected to hit directly at the expectations of speedy software cycles, thereby easing pressure on software developers to deliver massive amounts of code in tight timespans.

According to VentureBeat, “While Sourcegraph has always allowed its users to search for every repository file and line of code that might need to be modified as part of a large-scale refactor, it didn’t previously offer any functionality to execute this — and that is what batch changes is all about. “Sourcegraph now enables companies to automate and support large-scale code changes end-to-end,” CEO and co-founder Quinn Slack told VentureBeat.”

The objective behind several of these ‘batch changes’ is to optimise coding for larger firms with multiple departments and developer teams, i.e. larger repositories of code. Whilst companies often develop similar boilerplate codes to be reused by all teams involved, modifying the code has always been a ‘slow and painful’ challenge. Such batch changes could thus prove to be a much more prudent means of handling big code; much more inexpensive than firms such as Facebook or Google who spend a lot of money in developing their own proprietary coding platforms.