JFrog Ltd.

10/18/2021 | Press release | Distributed by Public on 10/18/2021 07:33

GitLab vs JFrog: Who Has the Right Stuff?

Like the historic space race, the competition to plant the flag of DevOps is blasting off. According to market intelligence firm IDC, global business will invest $6.8 trillionin digital transformation by 2023. Yet research also suggests that 70 percent of them will fail to meet their goals.

JFrog was the first company to offer a universal, hybrid, end-to-end DevOps platform. Powered by Artifactory, the industry standard in binary repository management, the JFrog DevOps Platform provides a comprehensive set of transformative solutions for automation, package and artifact management, security, and distribution to the edge.

GitLab also sports the DevOps platform label, extending their popular version control system (VCS) for source code with a large and varied portfolio of solutions for DevOpstasks. Brought together by a common front-end UI, GitLab seeks to build a single "DevOps platform" that can replace DevOps toolchains.

As JFrog has understood since its 2008 founding, GitLab recognizes that "[organizations] want faster cycle time, improved efficiency, and reduced risk."

Let's take a look at each to see who has "the right stuff" to help you succeed at those DevOps goals.

Platform or Portfolio

GitLab makes an excellent VCS - as even some of our developers at JFrog who have used it will attest. GitLab's expertise at the leftmost side of the software development lifecycle (SDLC) has helped them produce innovative features - such as codebase projects- to support design and coding.

Through a uniquely transparent, open-source development model, GitLab has rapidly expanded its subscriptions to include tools that help automate some rightward tasks of the SDLC such as CI/CD, package management, static testing, dynamic testing, and more.

Maturity?

While impressive in scope, most of GitLab's offerings are currently far from mature. Of the more than 40 rightward items not related to source code or CI/CD in GitLab's maturity chart, only 3 are "complete." Nearly half of those remaining are still "minimal."

GitLab Maturity Status as of September, 2021 (source: GitLab)

Unified?

The GitLab suite provides a common user interface, and they stress as a core value that no feature is considered complete unless it can be controlled through the UI.

While this provides the appearance of a unified platform, it obscures the reality that many of the GitLab tools are not genuinely integrated with each other. Behind the UX curtain, few of the pieces interoperate smoothly - often not even sharing a common metadata model. Even as DevOps seeks to break down silos, many of GitLab's DevOps and DevSecOps solutions remain siloed from each other.

Vision?

Successful DevOps doesn't come from solving individual problems one at a time. Success comes from applying the right methodologies that have been proven to work by hundreds of other organizations. Your DevOps tools need to enable the best practices that help you accelerate quality software releasesand deliver them safely to customer's desktops and devices.

While GitLab seeks to provide end-to-end tools for DevOps, they do not offer an end-to-end method for DevOps success.

When it comes to DevOps, a portfolio is not a platform. And our long experience in DevOps has shown that digital transformation success lies in managing your binaries, not just your code.

The Right Stuff: Development vs Delivery

Our thousands of successful customers have proved one truth: Achieving DevOps success is all about your binaries.

Software development is about source code, and quality code comes from empowering smart people who know how to write it.

Software delivery is about binaries - ensuring quality builds and getting them swiftly into your customers' devices. Quality binaries come from smart systems that know how to manage and distribute them.

As the vendor of a leading VCS, GitLab is expert at the necessary task of enabling software developers to create and manage source code. But the aim of DevOps is to achieve deliveryof high-quality releases from developers at top speeds.

A binaries-centric approach - JFrog's expertise - is the only way to successfully automate a modern organization's software lifecycle that ensures trust and speed of delivery.

To get that done, you'll need to be able to achieve these critical milestones:

Release Faster

Accelerating releases requires an accelerated binaries workflow that creates trust.

JFrog GitLab
Package Types 30+ 11+
Proxy Repository Caching ✅ Docker Hub Only
Automation: Natively integrated CI/CD ✅ ✅
Build Metadata for SBOM and Traceability ✅ X
Build Promotion for Release Staging ✅ X
Advanced Query Language ✅ X
Distribution Solution ✅ X


Package Types

JFrog internal data shows that, on average, an Artifactory installation maintains repositories for at least 7 distinct package types. Enterprise-level users demand even more, with half of those using 12 or more package types.

By these measures, GitLab's current local repository support for 11 package types offers a solid start - although far fewer than the over 30 package typesnatively supported by Artifactory. Like Artifactory, GItLab also provides a Generic repo type, enabling users to centrally manage additional file types that are part of their releases - such as images, zip files, docs, and more.

Language or OS JFrog Artifactory (partial) GitLab Package Registry (entire)
Java npm/Yarn npm/Yarn
Javascript Apache Maven Apache Maven
Javascript Gradle Gradle
Javascript Bower
Python pip/twine pip/twine
.Net Nuget Nuget
Golang Go modules Go modules
Docker Docker Docker
Helm Helm charts -*
C/C++ Conan Conan
iOS CocoaPods
PHP PHP Composer PHP Composer
Ruby RubyGems RubyGems
Rust Cargo
Linux, Yum RPM
GNU Linux Debian
Hashicorp Terraform (soon) Terraform

* With Helm v3, GitLab users can use the Container Registry to store Helm Charts. However, due to the way metadata is passed and stored by Docker, it is not possible for GitLab to parse this data and meet performance standards.

Proxy Repository Caching

Over 92 percent of applications use open source code from public repositories, which can represent a commonly estimated 60-90% of the code in each app.

Artifactory's remote repositoriesenable you to cache those open source packages in a proxy repo, enforcing version immutability, providing local speed and ensuring against any connection outage. You can logically combine these with local Artifactory repos as a single virtual repository, providing convenience to developers and governance by administrators over what can be accessed.

GitLab's Dependency Proxy provides a similar function - but it's currently available only for the Container Registry, and only proxies Docker Hub.

Build Metadata for SBOM and Traceability

Every build of an application is composed of many, many artifacts - the building blocks of software - from your packages and configuration files to the binaries that are the deployable runtime components of your application.

When you achieve the DevOps goal of multiple builds per day, it adds up quickly. Our enterprise customers each maintain an average 20 million unique artifacts, adding 130% more each year.

Artifactory stores extended metadata - what we call "build info" - with every build you make, from any build tool, linking to the package metadata of your open source and proprietary dependencies along with build artifacts and environment settings. With detailed build info, you can trace every deployable binary back to where it came from and out to every place it's been staged for service.

Your build info is also the basis of a Software Bill of Materials (SBOM) - a machine-readable inventory detailing all the items included in an application and their origin - for every release put into production or delivered to a customer. With a growing number of governments and regulated industries requiring an SBOM to help combat cyberattacks, Artifactory is your rapid turnkey solution for compliance.

GitLab has no comparable function to Artifactory's build info. GitLab CI/CD offers some analog to a "build" which they call deployments. This enables you to store a record of the build event with your source code, along with where it was (or, for manual deployments, will be) deployed. But this data describes the event, not the deployed binary, and does not include any of the metadata required to produce an SBOM or replicate a deterministic build. Moreover, by being an exclusive feature of GitLab CI/CD, this tracking mechanism cannot be extended to your legacy CI/CD pipelines, such as Jenkins.

Instead of metadata, GitLab's dependency list function works only with its dependency scanning tool to produce dependency data by parsing the source in the GitLab VCS. Consequently, GitLab cannot produce an absolutely reliable SBOM from your deployable binary - they can only reconstruct what is likely to be in your binary as deduced from the source code.

Build Promotion and Release Staging

Every new software version must pass several quality gates in an SDLC. But what passes through these gates makes the difference between a speedy or a plodding path to release.

Artifactory enables build promotion, in which an immutable binary runs through the entire SDLC. With a repository for each SDLC stage, a build with its metadata can be promoted simply by shifting it to the next repo in sequence.

In this "build once and promote" method, the same build is evaluated at every stage, assuring absolute consistency through the DevOps pipeline. Once free of having to perform their own deterministic builds, teams can apply the hours they recover to conducting more exhaustive tests and delivering feedback more quickly.

Build promotion is built into Artifactory's DNA - a simple API action can promote a build from one repo to the next, and Artifatory's checksum-based storage guarantees that a given build and its metadata are identical in all repos where it's stored.

While GitLab provides repos for generic artifacts like release binaries and container registries for Docker images, GitLab simply doesn't provide built-in support for build promotion. While GitLab CI/CD does support recording deployments to defined environments in your GitLab projects, this feature is unavailable to use with other automation tools.

Distribution Solution

From JFrog Cloud repositories, you can distribute high volumes of software across multiple locations through Amazon's CloudFront CDN solution. JFrog Distribution provides enterprise-ready solutions for the secure delivery of validated applications and updates to the servers, desktops, and devices where they can be put to use. From the binaries in your Artifactory repositories, create signed release bundles for delivery to Artifactory edge nodes.

GitLab offers no distribution solution at all, falling far short of their claim to be an end-to-end DevOps platform.

Connect and Automate

To be versatile, your DevOps platform needs to be able to support all the technologies that your development teams use, to merge seamlessly with the tools you need.

JFrog GitLab
CI/CD Automation ✅ ✅
Pre-built (native) steps & extensions ✅ X
Templates ✅ ✅
Simple CI/CD Creation ✅ ✅
Conditional Execution & Complex Pipelines ✅ ✅
Pipeline Editor X ✅
Signed Pipelines ✅ X
Linux Build ✅ ✅
Windows Server Build ✅ partial
Auto-Scaling Build Infrastructure Support 3 2
REST APIs ✅ ✅

CI/CD Automation

Much like their VCS offering for source code, GitLab's CI/CD is mature and widely used. GitLab pipelines are created through descriptions in YAML, where developers can define pipeline stages and actions.

Similarly, JFrog Pipelines also uses YAML to define pipelines. But there are some important differences:

  • Native Steps
    While GitLab's CI/CD is strongly integrated with GitLab source code repositories, each tool for build, test, and deployment must be invoked through their individual command line interfaces in shell scripts.

    JFrog Pipelines is naturally integrated with all mission-critical parts of the JFrog Platform - not just Artifactory, but also Xray and Distribution. Pipelines' native steps reduce out-of-the-box effort by enabling many common actions (such as Docker image builds, security scans, or release bundling) through pre-built steps defined in declarative YAML.

    Pipelines native steps can be mixed with general-purpose steps to execute shell commands, or extended by creating your own Pipelines Extensions that can hide low-level complexity and be shared across a team, department, or organization.
  • Simple CI/CD Creation
    GitLab's Auto DevOps feature helps to get started with CI by automatically setting up pipelines and integrations for you based on your source code. JFrog Pipelines reduces complexity significantly through declarative native steps and custom extensions, so you can focus on what you want done rather than how to get it done. Additionally, you can jumpstart creation of CI/CD with Pipelines Templatesto quickly create pipelines for common operations. We provide several built-in, and you can add your own to help ensure your teams follow best CI/CD practices.
  • Complex Pipelines
    GitLab's Parent-Child pipelines enable pipelines to behave more dynamically, automatically choosing to start (or not start) sub-pipelines based on the outcome of another.

    Similarly, in Pipelines any pipeline or step can be configured to triggeron a variety of events, including the successful completion of another pipeline's output, or a personnel-controlled approval gate. Pipelines' Graph viewprovides a combined, real-time status view of all interconnected pipelines and resources - your "Pipeline of Pipelines" - to understand dependencies between them.
  • Signed Pipelines
    As part of the unified JFrog Platform, Pipelines creates a blockchain-like, cryptographically-signed ledgerfor each run of a pipeline. It also adds enhanced metadata to builds and release bundles including a link to the run that generated it. Customers can then block downstream actions such as build promotion, release bundle creation, deployments, etc … if the builds/release bundles cannot be validated as being created by the linked run. This "zero trust" approach provides an additional layer of security by guaranteeing the authenticity of packages and making pipelines tamper-proof.

APIs, CLI, and Integrations

GitLab and JFrog each provide REST APIsto enable integration. Review them both - we think you'll find JFrog offers a more comprehensive set for versatile automation.

The JFrog CLI offers a ready way to access your Artifactory repositories from the command line or to automate from a shell script. GitLab's subscriptions provide no CLI at all.

GitLab CI/CD is tightly coupled with GitLab source control repositories. While the JFrog Platform provides end-to-end DevOps with Pipelines, you can also use Artifactory with CI/CD toolsyou might prefer - whether that's Jenkins, CircleCI, or even GitLab CI/CD. JFrog provides several integrations out-of-the-box, or choose from a large family of technology partner integrations.

Protect Your Business

Practicing strong DevSecOpsmeans having the best risk data and being able to interpret results to maintain safety and regulatory compliance. It also means enabling everyone along the software production process to be aware of security.

JFrog GitLab
Software Composition Analysis (SCA) Scanning ✅ Limited
Container Scanning ✅ Limited
License Compliance ✅ ✅
Automated Policy Enforcement ✅ X
Impact Analysis ✅ X
IDE Integrations to shift-left security ✅ X

SCA Scanning

GitLab Dependency Scanningtool is tightly integrated - and can only be used - with GitLab source control repositories and GitLab CI/CD to identify vulnerable open-source dependency references in source code. It scans source code from within a CI/CD pipeline; information about vulnerabilities found and their severity is reported in the merge request, so a developer can act to remediate.GitLab does notscan packages in GitLab Package Registry.

JFrog Xray performs deep-recursive SCA scans on your packages and binaries, leveraging your Artifactory metadata to identify the open-source components directly from your builds. This provides greater certainty, ongoing vigilance, and the ability to flag zero-day vulnerabilities in binaries already deployed.

To help shift-left security, developers can also invoke Xray to scan dependencies in the source code in their local directory, enabling them to remediate even before committing code to a branch.

Container Scanning

GitLab container scanning uses the open source Trivy engine as of GitLab 14.0. Container scanning is currently not integratedinto the GitLab Container Registry flow - Docker images can only be scanned through a separate job in GitLab CI/CD. Images pushed to the registry are not automatically scanned.

JFrog Xray can be configured to scan Docker images (including OCI-compliant images and Google Distroless images) in a registry continuously for both vulnerabilities and license policy violations, just as it can any of the 18 package types Xray supports.

Automated Enforcement Policies

GitLab's Dependency Scanning, Container Scanning, License Compliance and other security tools all provide reports that must be read, evaluated, and acted upon by a human operator - a set of high-friction manual steps that inhibit speedy software delivery.

JFrog Xray empowers security teams to configure rules and policiesfor vulnerability severity and license conflicts, and set up automated watchesto detect violations and enforce those policies after a scan. Through JFrog partner integrations, you can also report violations through analytic tools such as Splunkor DataDog, or alert teams with incident reports through PagerDuty, Slack*, or MS-Teams.*

*Currently in Beta

Impact Analysis

JFrog Xray reveals the risk impact to your entire binaries ecosystem through impact graphs that show the full scope of any vulnerability or policy violation throughout your inventory of builds. GitLab provides no comparable facility.

IDE Integrations

With any of the Xray plugins for popular IDEs, you can shift left securityawareness to developers, flagging and remediating vulnerabilities in OSS dependencies even before code commit.

As noted above, GitLab Dependency Scanning is driven through GitLab CI/CD. No IDE plugins for GitLab are currently available.

Scale to Infinity

Your business can't stand still; it must be able to seize every opportunity to grow globally. There are no limits and, no matter where you're starting from, your tools have to keep up. Your critical path operations must fit your needs today, but also enable business agility to meet your needs of tomorrow without interruption.

JFrog GitLab
Expandable High Availability ✅ ✅
Regional Geo-Replication ✅ Limited
Multicloud Offering ✅ X
Hybrid Solution ✅ Limited
Unlimited number of users ✅ X
Private Distribution Network ✅ X


Expandable High Availability

Both GitLab and JFrog support high availability(HA, also known as "clustered") deployments using multiple, load-balanced instances to help assure swift response time while enabling failover protection and zero downtime when performing upgrades.

Regional Geo-Replication

GitLab Geosupports limited site replication through unidirectional mirroring from a single primary GitLab site to read-only secondary sites. We consider this inadequate to support the way that global development teams collaborate.

The JFrog Platform supports a variety of replication topologies, most readily through Federated Repositories, an innovative bidirectional mirroring technology that empowers geographically distributed teams to collectively produce and share artifacts with their metadata. Within each federated repository, changes made to artifacts or to the repository configuration on one site are automatically synchronized to other member sites (up to 10).

Multicloud and Hybrid

GitLab hosts all cloud (SaaS) services on a single cloud platform, GCP in the U.S, making multi-cloud redundancy impossible. Although GitLab has both a SaaS service and self-managed installation option, these are separate and users cannot work between and across them.

JFrog Cloud (SaaS) is available for managed hosting on all major cloud providers(AWS, GCP, and Azure), empowering you to choose your cloud platform or maintain more than one for a multi-cloud strategy. All subscription levels of JFrog Platform (Pro, Team, Enterprise, Enterprise+) are available for self-hosting or on-premises and can be combined with any JFrog Cloud account to build an enterprise-ready hybrid system through repository geo-replication.

Unlimited Users

When your license fees are by the number of user accounts, that can significantly add to the expense of an expansion or acquisition. With JFrog's unlimited user licensing, that's not something you'll need to ever think about; add as many user accounts as your installation can practically support with no added cost.

Private Distribution Network

You can distribute releases from your Artifactory single-source-of-truth to Artifactory edge nodes, which can be connected into a multi-node topology to form a secure, high-speed, private distribution network (PDN) that spans the globe. Through JFrog's innovative peer-to-peer networking technology, you can overcome poor network connectivity, latency, and cross-border obstacles to deliver large, signed release bundles safely at top speed worldwide.

As noted above, GitLab provides no delivery or distribution solution.

Once You Leap Forward, You Won't Go Back

The strength of GitLab as a VCS and project manager is clear from the many organizations who rely on them for source code management. In fact, many of JFrog's customers do, using JFrog's universal CLI and APIs to integrate with the CI/CD of their choice, whether that's GitLab, Jenkins, or another tool. Of the customers listed in GitLab's recent Form S-1 filing, over half are JFrog customers as well!

With their broad portfolio of tools, GitLab's offering asks you to replace a best-of-breed toolchain with one from a single vendor. But unless that vendor can offer you a better chance to succeed, that's a risky bargain.

At JFrog, our mission is simple: To make every software creator successful by providing the best solution to deliver releases fast, securely, and continuously.

GitLab's ambitions are certainly large, according to their mission statement. "Our BHAG over the next 30 years is to become the most popular collaboration tool for knowledge workers in any industry."

Why wait three decades? The JFrog DevOps Platform, powered by the industry's most popular tool for binaries management, is available today and you can start for free!