No matter how many software developers you employ, development processes or cultures (such as DevOps or DevSecOps) that you adopt, sensitive technical data such as code, credentials or security infrastructure can still be prone to online exposure.
In this blog, we walk you through how modern software development practices lead to technical leakage, despite security controls in place to minimize this – and how security teams lack the visibility to detect it. Then, we’ll reveal some tips and tricks on how security teams can minimize their attack surface.
DevSecOps and The Evolution of Software Development
As customers, we can be a bit demanding when it comes to technology products. We want the latest products, the latest features – or the most recent versions of those. We’re not stuck for choice though. Rather, our menu of technology products is ever-growing. Companies are all racing to push out new products or better versions of those products to satisfy our digital hunger.
Companies are racing to push out product updates more than customers may realize. Research conducted by Google highlighted that elite performing organizations deploy software updates to their end-users, on average, multiple times per day and on the lower end of that up to between once a month and between every six months. Regardless of the speed at which software is deployed, something shared between these organizations is the ability to deliver software quickly – and on top of that value for the customers.
Several processes, tools, and cultures have evolved which facilitates the speedy delivery of products. In terms of processes, digital transformation has shifted development practices online, mostly to the cloud. Along with that, code platforms have proliferated, with a stronger preference to use code collaboration platforms to develop and deploy software. Then, throughout the development lifecycle, to ensure software teams collaborate more effectively and securely, development practices and cultures have evolved, though adoption varies, commonly known as Developer Operations (DevOps) and Developer Security Operations (DevSecOps). It’s the combination of the above factors that have, respectively, transformed software development functions for good, driving operational efficiency, flexibility, and agility. However, despite the progress made, challenges still exist, particularly in terms of securing software.
In this blog, we’ll review how modern security practices, among other factors, increase the likelihood of technical leakage. Along with this, we’ll introduce you to software development processes and cultures, DevOps and DevSecOps, that attempt to minimize technical leakage exposure, through security controls – but, as we explore, even with security controls leakage is still possible.
Software development: Frameworks that redress security
Depending on the maturity or size of a software development function within a business, several software applications and cultures have evolved, to not only reduce friction between these teams but also to address security. To address the former, is the culture of Developer Operations (DevOps) and addressing the latter is the application and culture of Developer Security Operations (DevSecOps).
What is DevOps & DevSecOps?
The software and engineering function is broad in scope – and its application varies by organization. At a very high level, software development can range from software engineers, who build software, to platform and cloud engineers who may build infrastructure around it.
To bring these teams closer together, and drive frictionless collaboration, one commonly known culture exists known as Developer Operations (DevOps). Cross-functional collaboration creates many process improvements, such as increased flexibility and agility, however, security often becomes an afterthought, as it is unable to keep pace with deployment. By itself, security in DevOps increases friction – slowing the product’s time to market.
De-prioritizing security can create some serious problems for software, though, which is already inherently prone to flaws, taking the form of either bugs or vulnerabilities. Not only do these negative occurrences make for unpleasant user experience – but they can, depending on the severity, allow an attacker instant access into an organization. As such, software demands IT and network security to work collaboratively to address these types of flaws, quickly.
DevOps versus DevSecOps
A step on from DevOps is the application and culture known as DevSecOps. This is a term that embodies a practice as well as a culture. When adopted, DevSecOps embeds security from start to finish within the software development lifecycle, while aiming to minimize time to market. As we’ve argued before, adopting DevSecOps and its programs can result in happier customers; there’s increased uptime, better support, and less lead time between fixes.
DevSecOps Toolchain: Intro to security controls
A DevSecOps toolchain is a great way to visualize how security can be embedded into the development lifecycle, continuously. A few interpretations of the toolchain exist, varying from Gartner to SANS. For reference, The SANS DevSecOps toolchain [Figure 1] outlines several security controls beneath each phase of the life cycle.
Security controls aim to improve deployment efficiency
According to SANS, security controls bake security into the toolchain, attempting to minimize the frequency and intensity of bugs and flaws, more easily. However, not all security controls are airtight – and must be managed appropriately.
However, the adoption of DevOps and DevSecOps varies by organization. Even with security controls listed outlined above, technical leakage can still persist, as a result of poor security practices. Here are key challenges affecting this space, regardless of whether DevOps or DevSecOps has been implemented.
The Digital Risks in Software Development
You may be wondering, why is Digital Shadows talking about software development? What has this got to do with digital risk? By definition, digital risks are unwanted exposure of data (company assets) across the open, deep, and dark web. Those risks can take form in the loss of sensitive corporate data, disruption of identity, violation of privacy laws, and damaged reputations.
Adding to the dynamic list of digital risks, Digital Shadows has observed the exposure of data emanating from the software development functions of organizations.
Who should own the problem?
Though the problem may emanate from software development functions, most security teams may now have to monitor and track the risks that threaten product success. In the ever-growing list of responsibilities that fall under their remit, security practitioners, depending on the type of organization and industry, may now have to proactively detect whether an employee – or perhaps a contractor or supplier – has inadvertently been exposing sensitive code or publicly committing to code repositories. For organizations, those risks can be deemed technical leakage – and impact organizations to different extents.
Concerted effort to minimize risks
On the other side of that, you also have the code collaboration platforms, which too can exacerbate the problem, if privacy controls aren’t enforced. In 2018, GitHub began offering token scanning services to prevent the accidental exposure of technical credentials. However, this isn’t just about GitHub; this also extends to software platforms where developers seek and share technical advice, such as GitLab, BitBucket, Azure, and Stack Exchange. These services have a joint responsibility to safeguard their customers, as best as they can. Exposure needs to be minimized from all sources – and no one party is responsible, though they can exacerbate the problems if unmanaged.
Together, organizations should also act to prevent such exposure, by strengthening internal security policies and educating consumers on the risks of exposure. Without such guidance, exposure of secrets and sensitive code is inevitable. On the organizational side, monitoring for technical leakage need not create another burden, though it may seem like it. Now, Digital Shadows makes it even easier for organizations to detect potential technical leakage – which we’ll go into depth later. But first, it’s worth diving into some of the key risks we have seen.
4 Key Risks impacting Software Development
- Leaked credentials: Exposure of Secrets
One of the biggest challenges of software development is when secrets become exposed by developers. Secrets (in software terms) are forms of digital authentication, such as passwords, API, access keys, and more. Secrets should even not be stored in a private code repository, and they definitely should not be in a public repository.
Unfortunately, in reality, this is worryingly common, meaning those crown jewels are fully accessible to everyone. Researchers at North Carolina State University found “that not only is secret leakage pervasive — affecting over 100,000 repositories — but that thousands of new, unique secrets, identified as AP and cryptographic keys, are leaked on Github at a rate of thousands per day.” Further highlighting how exposure on code repositories, such as Github, was a growing concern raised by users on Twitter highlighting how passwords could be searched across Github – imploring Github to introduce new features, instantly, to address such exposure.
There are also open-source tools, such as Trufflehog, a tool designed to search git repositories for secrets and Git Hound, which offer Github scanning tools, to detect sensitive data and prevent sensitive data from being committed (publicly posted).
What might appear to be an innocuous post seeking advice, or an inadvertent public commit or post, actually provides attackers with a goldmine of information.
- Leaked sensitive technology
While the exposure of technical credentials can be damaging, an equally important area to consider is whether the company’s infrastructure, which may hold data, is exposed online too. Knowledge of infrastructure could allow an attacker to perform reconnaissance on the organization, and build a deeper understanding of the target.
We know from our own monitoring that attackers actively seek this type of information. For example, Figure 2 illustrates a post made by an actor to the Russian cybercriminal forum. In the post, the actor allegedly details a list of company subdomains, which then reference ElasticSearch, indicating that these organizations use ElasticSearch as a form of data management and/ or storage. While there is no indication that these instances are exposed, this type of enumeration of technical information is incredibly helpful for malicious attackers’ campaigns.
- A growing number of breaches attributable to erroneous configurations
Depending on the organization, one of the core functions of DevSecOps is to oversee and manage IT infrastructure, which extends to configuration management of all servers in local, staging, and production environments, and ensuring that they are kept in sync and consistent.
Verizon’s latest research, 2020 Data Breach Investigations Report: Official, highlighted that misconfiguration errors were increasing. In particular, databases or file storages not being secured and directly exposed to a cloud service.While Verizon cited system admins setting the storage to the public, research independent of Verizon delved deeper into the causes of misconfigurations via a survey. Respondents cited a lack of awareness of cloud security and policies (52%), a lack of adequate controls and oversight (49%), too many cloud APIs and interfaces to adequately govern (43%), and negligent insider behavior (32%).
Similarly, Digital Shadows’s 2019 research, Too Much Information: the Sequel, highlighted how 2.3 billion files exposed through misconfigured devices, resulted from error and other factors.
These research pieces highlight how data can easily be exposed by weak to poor security controls or insufficient security awareness in place.
- Attackers picking on poor security practices
Often with data exposure, it’s worth asking “so what?”. So my files have been exposed, what now? A developer has posted to a public code repository, what’s the risk?
Unfortunately, such data can often fall into the wrong hands, which could cause significant damage. Here are some examples:
Data Extortion attempts
In 2017, activity was discovered accessing unauthenticated MongoDB installations and replacing their contents with a ransom note, usually containing an email, to force the victims to pay up in return for their sensitive data. Digital Shadows took the research further, concluding that the attacks were extortion rather than ransom since most campaigns did not appear to even attempt to view or backup the data before deleting.
In 2019, Digital Shadows revealed how threat actors were actively attempting to exploit exposed data: 17 million files across online file repositories, which are often used for backing up data, had been encrypted by ransomware, 2 million of them linked to “NamPoHyu”, a variant of the “MegaLocker” ransomware. (See more in our report: Too Much Information).
In 2018, an attacker located an AWS credential within code in a private repository for Uber Engineers on GitHub. Though the repository was private, it is thought that the attacker either brute forced or password guessed the credentials – allowing them access to the app’s databases, stealing personal information on 57 million passengers and drivers – information including names, email addresses, and phone numbers. Ensuring data is protected doesn’t just mean the databases aren’t publicly available – but that the access to it has sufficient authentication practices in place too.
According to research by Vinny Troia, GnosticPlayers were able to identify valid developer accounts by credential stuffing the HTTP-based API authentication, adding their SSH keys to the developer’s accounts using GitHub’s command-line tools, allowing the attackers to take advantage of an oversight in Github configuration.
Threat intelligence for DevSecOps: Minimizing the attack surface
There is a wealth of security controls that can assist with tackling these challenges head-on – and several products that map to it too. Now we won’t go into each of them – but one security control, that can benefit securing software development in particular is Threat Intelligence.
Threat intelligence can serve security teams by informing threat modeling and security architecture processes. In terms of threat modeling, threat intelligence can be applied to the pre-development phases of creating or making changes to code, and SANS recommends applying threat intelligence to address these types of questions:
- Are you changing the attack surface (new entry/exit points, new user role…)?
- Are you changing the technology stack or application security controls?
When it comes to security architecture, threat intelligence – specifically vulnerability and exploit tracking – can help organizations understand if their software is vulnerable to an attack. Threat intelligence is capable of so much more though and here’s where Digital Shadows fits in.
Threat Intelligence: SearchLight for DevSecOps
We have recently bolstered our technical leakage product offering. In addition to monitoring for code exposure or secrets across public repositories, organizations can now track, with the “unauthorized code commit alert”, their employees’ corporate activity on public code repositories. Identifying whether employees have unintentionally committed to a repository, provides organizations with a quick, scalable way to preemptively catch leakage before it becomes a serious threat. You can read more about the Unauthorized Code Commit alert here.
Furthermore, to stay up to date with the latest risks impacting software, organizations can start by monitoring and tracking the latest vulnerabilities affecting them. The figure below is an example of how customers can search across the open, deep, and dark web for vulnerabilities to web applications.
The adoption of DevSecOps varies by industry – and the above threat intelligence tips may not apply to all, requiring tailored approaches instead. Industries that are likely to be impacted include: retail, financial services, technology, telecommunications, energy, and more.
Begin Bolstering DevSecOps with Outside-In Intelligence
Like traditional security, DevSecOps is currently being affected by the forces of digital transformation, which blurs network perimeters, meaning data is increasingly likely to be exposed online. On top of that, the nature of the software development industry, which demands rapid deliveries and multiple stakeholders working together on collaborative tools, has also increased the likelihood of sensitive data to be exposed publicly. To prevent the exposure of software online and to minimize threats to software from modern development practices, Digital Shadows recommends the following:
- Monitor for exposure of sensitive technology and code: Benefit from a set of free tools or paid tools to detect public code exposure – or sensitive company assets across public repositories, such as Github, BitBucket, or Gitlab. Searchlight, for example, detects technical leakage – and whether an employee has used their corporate email to publicly commit to repositories. Learn more about Digital Shadows capability here. Alternatively, free, open-source tools available include Git hound, which prevents sensitive data from being exposed or TruffleHog, which searches through repositories for secrets.
- Harness contextualized Threat intelligence: Stay informed with the latest threats, tools, campaigns, and news impacting your organization and security architecture.
- Tracking vulnerable software: By aggregating data from open sources, SearchLight gains a broader picture of your network over time. This enables you to prioritize securing your network assets that are most at risk from compromise and exploitation. We provide high priority alerts that relate to genuine threats to your network infrastructure, not a deluge of CVEs (Common Vulnerabilities and Exposures).
- Increase awareness: Individuals may not be clued up when it comes to securing technical data. Such a problem can be easily solved by better education and training around these risks.
- Ease of use: When it comes to software collaboration tools, such as Github or Gitlab, ensure security protocols are set to prevent activity from being posted publicly
If you’re interested in learning more about how to implement DevSecOps for your organization, check out our resources below.
- DevSecOps: Continued database exposures point to growing challenges
- Digital Shadows’ Data Leakage datasheet
- Digital Shadows’ Data Leakage Detection capabilities
- Detecting Unauthorized Code Commit – Customer Story