GitLab adds CI/CD integration with GitHub

GitLab 10.6 has been released, featuring new CI/CD integration with GitHub and further integration with Kubernetes. With GitLab CI/CD for GitHub, developers can create a CI/CD project in GitLab and connect it into GitHub.

According to GitLab, while it already has received position feedback from its built-in CI/CD features, the company felt GitHub integration was a huge piece missing to its portfolio. GitLab also added the ability to integrate CI/CD with other repositories, such as BitBucket.

This new functionality was primarily designed for four audiences: open-source projects, large enterprises, GitHub users, and Gemnasium customers. Users with a public open-source project on GitHub will be able to take advantage of all of GitLabs highest tier features for free.

Large enterprises will now be able to use a common CI/CD pipeline for all of their different repositories. According to the company, many enterprises have wanted to standardize on GitLab, but could not because they had code stored in different repositories.

“Continuous integration and deployment form the backbone of modern DevOps,” said Sid Sijbrandij, CEO and co-founder of GitLab. “With this new offering, businesses and open source projects that use GitHub as a code repository will have access to GitLab’s industry leading CI/CD capabilities.”

GitLab is making the GitLab CI/CD for GitHub feature free for the next year. This means that anyone using GitHub for any reason will be able to take advantage of the this.

Following GitLab’s recent acquisition of Gemnasium, the new feature will allow customers that were using GitHub + Gemnasium to start using GitLab CI/CD without having to migrate code.

Other GitLab 10.6 features include the availability of Kubernetes Cluster Integration and GKE Integration, making it easier to users to use Kubernetes with GitLab. Kubernetes clusters can now be monitored right from within GitLab.

“With this release, we make it even easier for users to use Kubernetes with GitLab. You can now deploy a GitLab Runner to your connected Kubernetes cluster with a single click. You can also monitor your connected Kubernetes cluster from within GitLab itself. And you can now also see the IP address of an Ingress controller connected to your Kubernetes cluster, again, right inside GitLab,” the company wrote in a post.

Source: sdtimes

CollabNet announces new Agile and DevOps features in Winter 2018 release

CollabNet has announced the VersionOne Lifecycle and Continuum Winter 2018 Release with a number of new features and capabilities. VersionOne Lifecycle for Agile ALM is the company’s agile lifecycle management solution while VersionOne Continuum for DevOps is a continuous delivery solution that allows teams to speed up software deployments while reducing risk and ensuring quality.

“Our winter release continues to help organizations achieve end-to-end visibility so they can deliver software faster with reduced risk and confidence,” said Flint Brenton, president and CEO at CollabNet.  “CollabNet is committed to bringing enterprise software teams together unifying Agile, DevOps, and Source Control lifecycle management.”

As part of the VersionOne Lifecycle, the company has added a new Milestones feature. Milestones are designed provide more visibility into a team’s planning process by capturing and communicating key dates such as industry events. According to CollabNet, this will help teams keep track of targets, sync times and important events. Uses can also create and view milestones within their roadmaps.

Also included in the VersionOne Lifecycle for Agile ALM release are saved views for grid, and team transitions.

VersionOne Continuum now features enhanced integrations for TeamForge SCM. According to the company, this will provide a more streamlined interaction from Continuum to code repositories. The solution also features a new insights dashboard for collecting development data, an environments dashboard for viewing package revisions, and an encrypted plugin data for more security.

“We are excited to announce the 2018 Winter VersionOne Product Release. With this release, VersionOne launches new capabilities that provide teams with greater flexibility for planning and managing software delivery with confidence,” the company wrote in a post.

DevOps remains a competitive advantage

DevOps continued to dominate development teams and businesses throughout the year with organizations trying to reap the benefits. A Logz.io study found that despite DevOps being a well-known phenomenon, 50 percent of respondents are still in the process of implementing DevOps or have just implemented it within the past year.

In the past year, many software companies teamed with or acquired others to broaden their DevOps solutions. CA acquired Veracode in the beginning of the year to help add security to its DevOps portfolio. JFrog acquired DevOps intelligence platform CloudMunch in June. CollabNet and VersionOne announced a merger in August to bring agile and DevOps together. Perforce made a push into DevOps with the acquisition of agile planning tool provider Hansoft in September.

In addition to acquisitions, companies developed and released their own DevOps solutions from scratch throughout the year. Dynatrace started the year off with the release of UFOs, a status alert system designed to help DevOps teams get a better look into their deployment pipelines. Microsoft announced the release of Visual Studio 2017 in March with DevOps as one of its core pillars. VS 2017 included code editing, continuous integration, continuous delivery, and Redgate database integration. Microsoft continued its DevOps commitment throughout the year, ending with the preview release of Azure DevOps projects in November.

GitLab took a new approach to DevOps with the release of AutoDevOps in July, and shared its vision for Complete DevOps in October. AutoDevOps provides the ability to automatically detect programming languages and build an app in that language, and then automatically deploy it.The Complete DevOps vision combines development and operations into one user experience.

CloudBees released a DevOptics solution to provide metrics and insights between teams in August. Electric Cloud released ElectricFlow 8.0 with new DevOps insight analytics. Atlassian unveiled the Atlassian Stack and DevOps Marketplace to break down silos and accelerate DevOps adoptions, and brought DevOps workflows to scale with the release of Bitbucket Server 5.4 and Bamboo 6.2 in October

Companies also worked throughout the year to bring DevOps together with other software development approaches and tools. In January, CA released a report that revealed agile and DevOps worked better together than alone. Later in the year, CA released another study that found if businesses really wanted to boost their software delivery performance, they should combine DevOps with cloud-based tools. Scrum.org and the DevOps Institute teamed up on ScrumOps, a new approach to software delivery that brings Scrum and DevOps together.

One of the biggest new approaches to come out of 2017 was the idea of DevSecOps. DevSecOps is a new notion that bakes security into the DevOps lifecycle in order to find and fix security vulnerabilities earlier and all throughout the life cycle for faster, higher quality code.

Veracode started the year off with its release of Greenlight, an embedded DevSecOps solution that enables developers to identify and fix security vulnerabilities, and rescan the code to double-check issues are fixed. DBmaestro released the Policy Control Manager, a DevSecOps feature designed to eliminate risks, and reduce downtime and loss of data. In July, WhiteHat Security took a look at the success of a DevSecOps approach in its Application Security Statistics report. The report found critical vulnerabilities in apps were resolved in a fraction of the time it takes without a DevSecOps approach.

Other reports throughout the year looked at the challenges blocking DevOps: Redgate found databases were one of the most common bottlenecks for DevOps teams. Quali discovered infrastructure and fragmented toolsets were among the top barriers for DevOps adoption. And in a combined report, Atlassian and xMatters found successful DevOps implementations make the most out of culture, monitoring and incident management. The State of DevOps report conducted by Puppet along with DORA (DevOps Research and Assessment) found in order to achieve DevOps success, automation, leadership and loosely coupled architectures and teams are key.

According to Forrester’s software development predictions for 2018, DevOps tools will continue to proliferate and consolidate, and DevOps will drive the use of APIs and microservices.

Perforce acquires Agile planning tool provider Hansoft

Perforce today continued its push into the bigger software development life cycle and DevOps with the acquisition of Swedish Agile planning tool provider Hansoft. Terms of the deal between the privately held companies were not disclosed.

“We combine the developer productivity you’d find in GitLab or GitHub plus add scaling and repository management for version control and builds, and now have project planning and management,” explained Tim Russell, chief product officer at Perforce.

With its foundational Helix version control system and the recently released TeamHub CI/CD solution, Perforce is targeting mid-size to large organizations that are building technology products that generate revenue, Russell said. “Our differentiator is that no one has figured out scale,” he said. “Helix Core [server] has proven scale, and it extends to Git and projects in repository management.”

Perforce’s solution “helps companies transition to agile,” said Colleen Kulhanek, Perforce vice president of marketing, by offering support for the Scaled Agile Framework.

Hansoft has been a popular planning tool in the media, entertainment and gaming industries, Russell said, where development is rapid but predictability and linkage are difficult to attain. The acquisition “bodes well for other industries seeking predictability and [quick] iterations,” he said.

Russell added that the company thinks of itself now as Perforce 2.0. “We’re not the same company we were. Our portfolio is different, and the value we deliver to developers is different.”

DevOps. Scale. Modern development practices. That, Kulhanek said, is where Perforce is focused going forward.

For every network upgrade, hardware installation, cloud rollout or software update, there are countless tales of IT projects gone awry. As a result of poor project management, it’s all too common for projects to go over budget and past deadlines, leaving everyone involved frustrated and costing an organization significant resources.

To help PM’s avoid some of these headaches and help every project run a bit more smoothly, the following six elements are key for effective project management.

  1. A shared goal and vision – It’s motivating and inspiring for the team to know the big picture of what they are working towards, and it also sets the right expectations of deliverables. A common project management mistake is to not share the product vision and how the current project contributes towards achieving it. Ensure that team members understand the collective goals of a project and that everyone has bought into the overall vision at the outset.
  2. Team level project inception – The entire team must understand roles, responsibilities and deliverables. A kickoff meeting helps with norming and expectations, and doing this automatically makes the team more self-dependent and self-organized, while also instilling a higher level of accountability and ownership within team members. To the contrary, an ad-hoc start to a project leads to disconnected team members without common goals.
  3. Communication – There is nothing worse than a team member saying, “I didn’t know that.” It’s key to find a balance of team level communication using a mix of methods like emails, memos, phone calls, team-meeting, etc. to ensure that everyone is getting the same message, at the same time. Using technology for online meetings for remote members can also be helpful to build team connection and morale.
  4. Use of a common and shared process – Too much process causes frustration and makes people inflexible, but a lack of it makes the team dysfunctional without any clear understandings. It is highly important to norm on processes with which the entire team is on board in order to achieve that delicate balance between too little or too much process. Sometimes even little things, like no meetings between 1-3 p.m., help team members better achieve their tasks. The nuance here is that these standard operating procedures will be different for every team and every project.
  5. Gain visibility into other projects with dependencies – Identifying and managing dependencies is as important as managing one’s own project. Timelines and deliverables of a dependent team will govern your own project timeline, so it becomes crucial to have a visibility into the health of dependent projects. It is good practice to identify these dependencies early in the project kickoff process; building a dependency map for a project is as important as establishing scope for a project.
  6. Customer/stakeholder engagement – The best customer experience is one which is continuously evolving. To achieve this, it is important that the customer or representing stakeholders be a part of the development process via scheduled demos. Engage with customers/stakeholders early enough to establish a feedback loop. This will allow the stakeholders to know what is being delivered and allow them to give feedback for improvement. The demos should be scheduled at regular intervals so a cadence is established in accordance with the project timeline.

Even the best project managers can sometimes get overwhelmed, especially as projects grow larger and more complex. While every project and team will be different in its goals, procedures and challenges, project managers can set up their teams for success every time by adhering to the six principles discussed here

GitLab adds Auto DevOps, other features to version 10 of development platform

GitLab today is releasing version 10 of its development platform, with features for Auto DevOps, chat, issue boards, CI/CD and monitoring.

The release is the 71nd GitLab has made on the 22nd of the month since it began doing so in 2011, and CEO Sid Sijbrandij proudly stated the company has never missed a release.

“We started offering version control and issue tracking, and now we’re integrated for the whole software development life cycle,” he said. “It also contains a Docker container registry, so now you CI system is aware of the container system, and you don’t have to pass permissions around. You can go from an idea to getting it out there all within one application.”

The Auto DevOps feature enables developers to push code, and the automated software creates a project, figure out how to build and test the code and give you more information about the quality of the code, Sijbrandij said. It also helps you transition to Kubernetes, or cloud-native infrastructure, without having to figure everything out for yourself, he added. As for assessing the quality of your code, “We leverage open-source tools, and leverage the work of Code Climate, which packaged up those tools and open-sourced that work. Code Climate engines give results back to the user that could be, “Hey you made a change, this is how the quality of the code got better, and how it got worse.”

This release also introduces group-level Issue Boards, which can elevate Kanban boards, for instance, from individual projects to be able to visualize across projects – especially important in the world of microtasking and microservices development. The capability also offers the ability to lock an issue, and provides better support for Atlassian’s Jira issue tracking tool. “We have an alternative for Jira, but we don’t want to force people to use it,” Sijbrandij said. “By Q4, we’ll have better support for Jira than Atlassian does,” he predicted, adding that a Jira importer will be finished soon.

Aside for many performance improvements, the last of the big new features designed to help organizations move quickly from idea to production is a completely new interface. “As the scope of GitLab explaned, we needed to make a big change to the UI. We did a lot of user research and testing, and this has been in the making for months. Responses have been very positive. It’s easier for users to locate where they are, and they can visualize a complex hierarchy, create groups of groups and now navigate seamlessly through all that.

“We’re built for the enterprise,” Sijbrandij added. “Our bread and butter are companies running internally. People who self-host Git host on GitLab, and they’re all adding features. Now it’s possible to have an integrated solution that still is best of breed. The contributions to GitLab CI, which is making it so popular, is not just about us; it’s about the open-source community.”

Sijbrandij also discussed the trends he sees dominating development in the very near future: Kubernetes, DevOps and microservices. “When you’re creating multiple projects, per project you can spend less time on maintaining or integrating the tool solution. In the past you’d have to ask IT to set up CI and CD, now you can’t. There are just too many projects. You need a registry to see how you make all these services work together.”

As for Kubernetes, GitLab “is all in. We’re turning a new corner. We want to make GitLab the development environment for Kubernetes.  We’ll see shift from VMs to container schedulers. Kubernetes is hard, but we want to make it easier. Just attach a Kubernetes cluster to GitLab and just go. We’ll get the proven standards in there and deploy your app with those.”

In today’s DevOps world, you need to automate the SDLC from start to finish, he said. “For every code change, a staging server will show the application in that state. It’s complicated to do without Kubernetes and GitLab. Automating the code review, the app and how it works makes automating more essential with microservices.”

He added that dependencies between projects become much more important from a management perspective. “You need visibility. We released a better view into the relationships between projects in CI. You can can triggers for actions in a service, or dependencies on other services. In GitLab, those are now visualized.”

CloudBees increases collaboration with new CloudBees DevOptics solution

CloudBees is committed to the Jenkins and DevOps communities, and today at Jenkins World 2017, the company announced new offerings that provide valuable insights and metrics in context between teams, applications and tools to identify ROI, improvements and increase collaboration.

One of the new offerings focuses on providing feedback about hidden bottlenecks and pitfalls in Jenkins-based continous delivery streams. Dubbed CloudBees DevOptics, it creates a holistic view of the entire software delivery process, according to a company announcement.

Key benefits of CloudBees DevOptics includes its ability to “provide a single source of truth,” so users can collect and correlate data across software delivery pipelines into one comprehensive live view. The solution also identifies improvements with end-to-end visibility into the software delivery value streams, it enables collaboration, and it assesses the overall productivity of a DevOps delivery process.

“DevOps has clearly gained tremendous adoption as the way to deliver software faster and align IT to the needs of the business,” said Harpreet Singh, vice president of products at CloudBees. “However, despite the investments made in DevOps, enterprises do not fully experience the benefits. This is because there is a lack of visibility in software delivery processes that inhibits making informed decisions.”

CloudBees aims to solve this visibility challenge because its solutions connect with an ecosystem of DevOps tools, according to Singh, and that means the company has the data to assemble the end-to-end software delivery picture.

In addition to CloudBees DevOptics, the company announced CloudBees Jenkins Advisor, “which analyzes Jenkins environments continuously, identifies potential issues and advises on corrective actions before they impact business-critical software delivery,” according to a company announcement.

CloudBees Jenkins Advisor is available in a self-service fashion to all Jenkins users, and it installs via a free plugin. Once it’s activated, it provides an automatic scanning of a Jenkins master to identify all issues and alert the user. The company also announced a few other offerings, like its new user experience for teams using Jenkins to accelerate continous delivery, and its new DevOps managed services program delivered by premium managed service providers.

Electric Cloud releases ElectricFlow 8.0 with new DevOps Insight Analytics

In order to automate data collection from the entire DevOps toolchain, Electric Cloud introduced ElectricFlow 8.0 with new DevOps Insight Analytics. This new solution provides teams with automated data collection and reporting to connect DevOps toolchain metrics and performance. It also provides visibility back into the business value and health or status of software releases.

According to CEO of Electric Cloud, Steve Brodie, Electric Cloud speaks to many companies that have 60 or more tools in their software delivery pipelines, which makes reporting “cumbersome and ‘fast feedback’ difficult for them to achieve,’” he said. With the ElectricFlow 8.0 release, executives and technical team members have the ability to collect and report on metrics from their myriad of DevOps tools, environments and processes involved, he said. Some of these tools include Jenkins, JIRA, HP ALM, and ServiceNow, which all provide visibility into the entire software development lifecycle.

ElectricFlow DevOps Insight is built on ElasticSearch and LogStash, it’s delivered as an integrated stand-alone BI server, and it can pull key metrics from any tools being orchestrated as part of the end-to-end delivery process. This includes user story tracking and build automation.

Teams also have access to a software development kit which extends the solution to capture custom metrics from new tools, according to the company.

ElectricFlow Community Edition with DevOps Insights can be downloaded and used for free for new subscriptions during 2017, according to Electric Cloud in an announcement.

The main goal of DevOps is quite simple: ship software updates frequently, reliably, and with better quality. This goal is somewhat “motherhood and apple pie,” since almost every organization will agree that they want to get there. Many will tell you they’ve already embarked on the DevOps journey by following some commonly followed frameworks, such as “CALMS.”

However, very few will express complete satisfaction with the results. After speaking to 200+ DevOps professionals at various stages of the adoption lifecycle, we found that organizations generally fall in one of three categories:

We were most interested in groups two and three since they were actually in the middle of their DevOps journey. When asked to better explain the challenges and roadblocks, here is what we found:
•68% said that the lack of connectivity between the various DevOps tools in their toolchain was the most frustrating aspect
•52% said that a large portion of their testing was still manual, slowing them down
•38% pointed out that they had a mix of legacy and modern applications, i.e. a brownfield environment. This created complexity in terms of deployment strategies and endpoints, toolchain, etc.
•27% were still struggling with siloed teams that could not collaborate as expected
•23% had limited access to self-service infrastructure
•Other notable pain points included finding the right DevOps skill set, difficulty managing the complexity of multiple services and environments, lack of budget and urgency, and limited support from executive leadership

Let us look at each of these challenges in greater detail.

#1: Lack of connectivity in the DevOps toolchain
There are many DevOps tools available that help automate different tasks like CI, infrastructure provisioning, testing, deployments, config management, release management, etc. While these have helped tremendously as organizations start adopting DevOps processes, they often do not work well together.

As a classic example, a Principal DevOps engineer whose team uses Capistrano for deployments told us that he still communicates with Test and Ops teams via JIRA tickets whenever a new version of the application had to be deployed, or whenever a config change had to be applied across their infrastructure.

All the information required to run Capistrano scripts was available in the JIRA ticket, which he manually copied over to his scripts before running them. This process usually took several hours and needed to be carefully managed since the required config was manually transferred twice: once when entered into JIRA, and again when he copied it to Capistrano.

This is one simple example, but this problem exists across the entire toolchain.

Smaller organizations get around this problem by writing custom scripts that glue their toolchain together. This works fine for a couple of applications, but quickly escalates to spaghetti hell since these scripts aren’t usually written in a standard fashion. They are also difficult to maintain and often contain tokens, keys and other sensitive information. Worse still, these scripts are highly customized for each application and cannot be reused to easily scale automation workflows.

For most serious organizations, it is an expensive and complex effort to build this homegrown “DevOps glue,” and unless they have the discipline and resources of the Facebooks and Amazons of the world, it ultimately becomes a roadblock for DevOps progress.

Continuous Delivery is very difficult to achieve when the tools in your DevOps toolchain cannot collaborate and you manage dependencies manually or through custom scripts.

Challenge #2: Lack of test automation
Despite all the focus on TDD, most organizations still struggle with automating their tests. If the testing is manual, it is almost impossible to execute the entire test suite for every commit, becoming a barrier for Continuous Delivery. Teams try to optimize this by running a core set of tests for every commit and running the complete test suite only periodically. This means that most bugs are found later in your software delivery workflow and are much more expensive to find and fix.

Test automation is an important part of the DevOps adoption process and hence needs to be a top priority.

Challenge #3: Brownfield environments
Typical IT portfolios are heterogeneous in nature, spanning multiple decades of technology, cloud platform vendors, private and public clouds in their labs and data centers, all at the same time. It is very challenging to create a workflow that spans across these aspects since most tools work with specific architectures and technologies. This leads to toolchain sprawl as each team uses the toolchain best serving their needs.

The rise of Docker has also encouraged many organizations to develop microservices-based applications. This has also increased the complexity for DevOps automation since an application now needs hundreds of deployment pipelines for heterogeneous microservices.

Challenge #4: Cultural problems
Applications evolve across functionals silos. Developers craft software, which is stabilized by QA, and then deployed and operated by IT Operations. Even though all these teams are expected to work together and collaborate, they often have conflicting interests.

Developers are driven to move as fast as they can and build new stuff. QA and Release management teams are driven to be as thorough as possible, making sure no software errors can escape past their watchful eyes. Both teams are often gated by SecOps and Infrastructure Ops, who are incentivized to ensure production doesn’t break.

Governance and compliance also plays a role in slowing things down. Cost centers are under pressure to do more with less, which leads to a culture that opposes change, since change introduces risk and destabilizes things, which means more money and resources are required to manage the impact.

This breakdown across functional silos leads to collaboration and coordination issues, slowing down the flow of application changes.

Some organizations try to address this by making developers build, test and operate software. Though this might work in theory, developers are bogged down by production issues, and a majority of time is spent on operating what they built last month as opposed innovating on new things. Most organizations try to get all teams involved across all phases of the SDLC, but this approach still relies on manual collaboration.

Automation is the best way to broker peace and help Dev and Ops collaborate. But as we see in other challenges, ad-hoc automation itself can slow you down and introduce risk and errors.

Challenge #5: Limited access to self-service infrastructure and environments
For many organizations, virtual machines and cloud computing transformed the process of obtaining the right infrastructure on-demand. What previously took months could now be achieved in a few minutes. IaaS providers like AWS have hundreds of machines with flexible configurations and many options for pre-installed OS and other tools. Tools like Ansible, Chef, Puppet help represent infrastructure-as-code, which further speeds up provisioning and re-provisioning of machines.

However, this is still a problem in many organizations, especially those running their own data centers or those that haven’t embraced the cloud yet.

We Need more from DevOps
A popular DevOps framework describes a CALMS approach, consisting of Culture, Automation, Lean, Measurement and Sharing. The DevOps movement started as a cultural movement, and even today, most implementations focus heavily on culture.

While culture is an important part of any DevOps story, changing organizational culture is the hardest thing of all. Culture forms over a period of time due to ground realities. Ops teams don’t hate change because they are irrational or want to be blockers. Over the years, they’ve taken the heat every time an over-enthusiastic Dev team tried to fast-track changes to production without following every step along the way.

Seating them with the developers might help make the work environment a little friendlier but it doesn’t address the root cause, no matter how many beers they have together.

Microsoft is gearing up for the release of SQL Server 2017 with its first release candidate. SQL Server 2017 RC1 indicates the development work for the upcoming version is complete and almost ready to come to market.

The latest version of the relational database management system will bring SQL Server performance and security to Windows, Linux and Docker containers.

Key enhancements in RC1 include: faster performance SQL Server on Linux Active Directory integration; TLS to encrypt data; machine learning services enhancements; SQL Server Analysis Services, SQL Server Integration Services on Linux and SQL Server Integration Services on Windows Server. In addition, Microsoft believes the latest version will perform queries up to 100 times faster than disk, better protect organizations, provide transformative insights with up to 1 million predictions per second, and enable users to build modern apps using any data, language or platform.

Since the latest version will bring support for Linux OS and containers running on Windows, Linux and macOS, Microsoft says SQL Server will be suitable for many DevOps scenarios.

To help users integrate SQL Server into their DevOps cycle, the company announced a new site: DevOps using SQL Server. The site aims to teach developers and development managers the knowledge necessary to integration SQL Server 2017 into the DevOps pipeline. SQL Server DevOps tools include Visual Studio SQL Server data tools, mssql-scripter, sqlcmd, and bcp. The site also includes demos, documentations, blogs, videos, and conference presentations.