The era of DevOps is upon us. DevOps is teaching us that when we do something repetitively, we make fewer mistakes. When applied to software, we find that continuous integration, deployment and feedback provided to developers is essential so that problems are rectified as quickly as they were introduced. This is the continuous software delivery goal.
Continuous practices require the decomposition of software development to manageable building blocks. Agile, open source, containers, and microservices introduce modularity in the application building process which compliments DevOps. Less monolithic apps, packaged in smaller pieces not only let organizations get innovative functionality into the market faster, they are easier to use with continuous practices and even make for happier engineers (which helps retain the best).
The landscape of vendors has shifted to reflect this new reality with over 200 vendors vying to win your business in the app development, software delivery and hosting categories.
At Codenvy, we are obsessed with helping development teams achieve faster software delivery. In this article, we will explore the nature of this continuous world and the impact workspace configuration is having.
Organizations are investing in Docker, Puppet, Chef, and Jenkins because these solutions are solving thorny problems around how to take code and then get it into the right environment in a consistent, repeatable way. These technologies are using recipes, automation, and management techniques to make this process not only continuous, but computerized, often performing their tasks repeatedly without a human launching the process.
This works well on centralized areas like testing, staging and production where these services are hosted and in control of the environments and resources. They have an awareness of the resource limits and coordinate these multitude of actions in such a way to optimize the delivery process to minimize time or failure rates.
But what about the developer’s workspace? Each developer needs a workspace to do their work, and these workspaces often require many of the environmental components that production systems require.
The workspace is the central location to perform a unit of work against a set of like-minded projects. Developers often need many workspaces, each configured differently to support the unique constraints of their project. Developers demand choice and flexibility, so they will continue to use their favorite IDE integrated into those workspaces directly. At the same time, each workspace is the central location that a developer uses to coordinate with their corporate toolchain or targeted systems.
The developer’s workspace is a miniaturized staging environment, connected to the same resources that an organization’s central systems are connected to, but with additional developer-centric services: debugging, code authoring, content assist, and file management. The system that hosts the workspace takes responsibility for a variety of infrastructure needs including toolchain integration, resource allocation, security, isolation, and reporting.
Today this workspace is almost always on the developer’s desktop. Their workspace is localhost. It’s commonly assumed that the workspace is bundled within the IDE, but the IDE is just acting as a smart client against the (assumed and embedded) localhost workspace.
Given this reality, it is probably not surprising that developers most commonly configure their workspace using manual wiki pages, which dictate a sequence of manual configuration steps that they often perform on localhost.
Sometimes they are able to automate these elements using Vagrant or Chef on localhost. But even with that automation, the developer quickly runs into a number of limits that cannot be overcome:
This is difficult to manage at scale when changes are frequent. And, it’s frankly not the Git-way.
It is not possible to remotely inspect the workspace on localhost or to query the image what its contents are. This leads to...
Organizations take great care to manage hosted source and issue management repositories, defining which resources have privileged access to intellectual property. But why bother when the first thing each developer executes is a repo clone onto their localhost workspace that isn’t monitoring or controlled by the organization? The code – once protected – can be copied to other storage devices. The security of your IP is only as strong as the security strength of the collective workspaces that integrate with your hosted systems. With developers demanding and receiving control over their environments, questions are raised about the appropriate way to affect governance while not sacrificing the productivity of individuals.
Localhost workspaces and images running within a localhost system are bound to the sharing rules dictated by the administrator of the computer running localhost – the developer.
There is no programmatic way to create, update, or destroy the workspace. Or even more powerful, once the workspace has been created, a programmatic way to orchestrate and automate services that are performed against the projects hosted within.
For developers that configure localhost directly, they have to jump through many hoops to have one set of configurations isolated from others. With developers working on many versions and projects simultaneously, they need isolated workspaces to stay productive. While images can create this isolation natively, it comes a cost.
Have you tried running 20 workspaces simultaneously each consuming up to 100 GB of RAM so that you can fluidly switch between projects without worrying about configuration isolation, waiting for an admin to provision new resources, or the endless whirl of your CPU fan from the constant thrashing due to resource swapping? It’s just not possible.
While many think of Codenvy as a browser-based IDE, which is something we do provide, we are primarily a developer workspace cloud. Codenvy workspaces are designed to be redundant, scalable, secure, and collaborative. They are designed to eliminate the restrictions that have been holding developers back without limiting their preferences on experience. We want these workspaces to be free of compromises.
All of the following are true with Codenvy workspaces:
Whether it is IntelliJ, Sublime, Eclipse, emacs, vi or our browser-based IDE, we are indifferent. Developers demand to choose their own experience so that they can make maximum use of their muscle memory or the plug-ins that their organization depends upon. We make this possible with a CLI that enables desktop cloning of workspaces for use by your local IDE and because...
Each workspace is accessible through an opinionated REST API. The API includes meta elements around the management of the workspace’s lifecycle, along with control over what is installed within the workspace and the actions that the workspace performs. There are over 100 possible methods to invoke with examples such as “create project in workspace by importing zip”, “get list of all runner processes for workspace”, “create new temporary workspace”, and “add member to workspace”.
Similar to the way a Chef recipe defines a server for execution or a Vagrantfile defines an image to be created, within Codenvy a workspace has a recipe that defines its structure. Recipes can be packaged into Factory URLs to enable repeatable re-creation of identical workspaces or they can be saved within your source code repository for versioning and inclusion as part of your configuration-as-code initiatives.
We take the various functions performed by users within their workspace and place them into microservices, which are executed within machines. Machines are lightweight services that can perform functions on your project in an isolated manner. We are able to place different machines onto different physical nodes, giving a single workspace access to a set of resources that far exceed what is available on localhost. We use Docker as our default machine implementation, but you can provide your own. You can even provide custom Dockerfiles that will allow us to use custom machines that are built on-demand to perform the functions in the workspace. When you run Codenvy On-Prem, you decide how many physical nodes to deploy into your cluster. With Codenvy SaaS, we provide these physical nodes on-demand.
Every developer, project, team, and company (or ISV!) are different. Depending upon the perspective of the organization, they may need to integrate special tools, override atomic microservices, or provide custom environments. Codenvy is based upon Eclipse Che, which provides an extension mechanism that lets developers create custom machines and extensions that modify the behavior of workspaces. Che extensions, whether they are client or server-side can be packaged into custom assemblies for Codenvy On-Prem. These extensions can not only modify a workspace, but can change the entire nature of the system itself, including the branding.
Codenvy is project-centric. Projects that are contained within a workspace are structured and opinionated based upon the type that they inherit. The system alters its behaviors by the type of project including altering the plugins that are loaded, altering the behavioral policies, and adjusting the logic for key microservices. For example, building a maven multi-module project requires a different procedure than that of a node project and the system adjusts for this. With customization, extension authors can create their own project types. Additionally, projects may have complex structures including sibling and parent-child relationships, each that bring an associated set of behaviors. Take, for example, the common scenario where a developer is working on a 3 tier application, for which the source code is in 3 different GitHub repositories and the execution is within two machines (app server and database server). In this scenario, a developer needs a single Workspace, three projects each mapped to a GitHub repository, and an environment that maps the projects to a set of 2 networked machines.
It is always better to build software together rather than apart. Workspaces, their projects, their files, and their executing machines are all represented with URLs which can be shared with others. Increase the membership of a workspace by extending access privileges to others. Multiple developers are permitted to work on the same projects (each in their own isolated copy), sharing the projects, files and resources allocated within. Make projects public or private to further restrict access.
Codenvy embeds a structure that inherently understands the iterative developer cycle of code authoring. At the core of development, engineers write code, build it, and then debug it – repeating this cycle with changes until the desired state of behavior is achieved. There are phases before this dealing with configuring the project and after this about moving the assets to another phase of the continuous integration cycle. With localhost workspaces, there was limited intelligence that was applied to this repetitive cycle and it became the domain of IDEs. IDEs would optimize this workflow for the technology each IDE specialized to, but limited by the resources and authorizations that it was offered as a client application that ran periodically. With hosted workspaces, they can – and do – run continuously, performing assist functions automatically based upon the state or events affecting the workspace.
This structure is baked into the system such that extension developers inherit these properties to their benefit allowing them to focus on implementing specialized logic unique to their project domain.
Codenvy lets account/owners, system/admins, and workspace/admins define access, behavioral and resource policies that the system will honor. These policies can be mapped against users or groups. This provides a variety of fine-grained controls that give those with super rights the ability to restrict the consumption of microservices, limit their access to certain behaviors, or to prevent access all together. For example, within your account, you can set RAM and Gigabyte hour caps on each workspace that will prevent new services from launching or even terminate existing ones if upper limits are reached. Beyond this configuration, we also track 100s of events across your workspace and those of all users, giving users and admins analytics to understand what people are doing within their workspace. Anyone can see their own analytics by accessing codenvy.com/analytics with their account credentials.
So is this model of workspaces right for you? Beyond the flexibility and power that you will get with hosted workspaces, you’ll also save money. Developers spent 100 gigabyte hours in 2014 on idle compute due to localhost. This problem is only going to get worse even though the cost of compute is declining
If you are tired of localhost being insecure, non-scalable, and unmanageable, it’s time for you to think about hosted workspaces with Codenvy. You can get started for free as you get 20 free gbh every month in our SaaS self-service workspaces. And Codenvy On-Prem is free for use within your own systems as well.
It’s the power, flexibility, security, and manageability of Codenvy workspaces that enables leading development organizations like Intuit, WSO2, SAP, CCCIS, YouthDigital, and Huawei scale their workspaces and achieve their continuous software delivery goals.