Do you really need to build your own Internal Developer Platform/Portal?  

Picture this. Let’s say you’re working in the IT division of an enterprise that has decided to embrace a cloud-first strategy to digitally transform its operations to meet its growth demands. 

Now, let’s say the IT leadership team that’s leading the effort has decided to use the opportunity to address some lingering IT-specific issues related to delivery speed, developer productivity, standardization, and technology governance.  

In addition, you’ve been tasked with implementing a platform to deliver on those IT-specific objectives. Great!  

This is all very exciting, but where to start? 

Defining an Internal Developer Platform/Portal (IDP) 

You join your tech friends at your favorite bar that night to celebrate your lucky draw and share the good news. Your friends are all super excited when they hear the news and raise their bottles in a toast. 

Now, it starts to get interesting.  

As it turns out, one of your friends has just returned from a conference where platform engineering and something called “IDP” was all the rage. He swears this is the answer to your challenge and encourages you to consider building an IDP.  

“Really?”, you ask. What the heck is an IDP anyway? 

First thing in the morning, you set out to answer that question and to determine how it would help you achieve the objectives you’ve been assigned. Most importantly, you want to establish how quickly you could build an IDP to impress your IT executives. 

After starting your search, you realize that the platform engineering and IDP rage your friend spoke about was, in fact, a real thing.  

It had even made it to the widely respected Gartner® Hype Cycle and Market Guide reports. The Gartner® “Innovation Insight for Internal Developer Portals” report asserts that “By 2025, 75% of organizations with platform teams will provide self-service developer portals to improve developer experience and accelerate product innovation.”

Interesting. 

Another thing you notice when researching is that there doesn’t seem to be much of a consensus regarding what the “P” in IDP means. Does it mean “platform,” as in “Internal Developer Platform?” Or does it mean “portal,” as in “Internal Developer Portal?” 

The best answer you find is a clarification attempt offered by Manjunath Bhat, Research VP, Software Engineering Practice at Gartner®. In his 2022 report, “A Software Engineering Leader’s Guide to Improving Developer Experience”, he stated that “Internal developer portals serve as the interface through which developers can discover and access internal developer platform capabilities.”

That’s a great place to start.

But the next question you ask yourself is, “what are the internal developer platform capabilities he speaks of?” 

Upon further research, you discover that the broad consensus answer to that question seems to revolve around the concept of “platform orchestration”. Specifically, capabilities that integrate multiple technologies and tools to automate and standardize repeatable processes – without impeding developers’ ability to innovate freely. Examples of such processes are CI/CD tooling and infrastructure provisioning. 

Aha! 

You’re about to create something that could be truly impactful for your company’s success in accelerating and standardizing its digital application deliveries.

“But, how to refer to the platform?”, you ponder.  

You decide to reach out to one of your highly respected mentors for some wisdom. She happens to have vast experience leading R&D organizations and consulting at several large enterprises. After listening to your exciting challenge, she points out how most of the enterprises she had worked with dealt with challenges similar to what you are trying to address.  

You exit the discussion by buying into the perspective that developer platform capabilities are a subset of the overall solution that a Platform Engineer enables and that the full value of the solution cannot be realized without the portal.  

You conclude it best to consider the IDP as an internal developer portal – given that’s how users access and experience the full value of the solution. 

“Life indeed can be truly beautiful”, you mutter to yourself, with a smile. 

Scoping the IDP “build” effort 

Now, you have a good idea of what your buddy was referring to as an IDP and have also ascertained through your research that platform engineering is the practice of enabling those capabilities. Naturally, you wonder why he claimed you could build it yourself, so you call to learn more. 

He points you in the direction of a platform called Backstage.io – an open-source framework for building developer portals that was created at Spotify. Apparently, this was a big part of the rage at the conference he just returned from.  

You figure it would be a great place for you to start building your own IDP, so you jump on the lead right away.  Your research quickly reveals that it offers the following essential internal developer portal capabilities – out of the box and through its free and paid marketplace plug-ins: 

• Security and access management (SSO, RBAC, Custom Roles, User mgt) 
• Software and service catalog 
• Software and service auto-discovery and metadata ingestion (from source code repositories, Kubernetes clusters, CI/CD pipelines) 
• Software and service catalog search and retrieval 
• Developer knowledge exchange and collaboration 
• Centralized access to tech docs 
• Scorecards and maturity assessments 
• Dashboards and reporting 
• Extensibility (customizable pages and views, plugin and API framework) 

That’s good, but you wonder where all the other capabilities you were expecting are based on the IDP research you just completed, especially those internal developer platform features. 

At this point, it occurs to you that the “build your own IDP” mission your buddy had suggested may not be such a slam-dunk after all. 

You embark on a gap analysis exercise to identify some of the other key IDP capabilities that are missing in Backstage. Features that you consider “must haves” in order to achieve the business impact you were aiming at.  The high-level list you come up with includes: 

Development Portal 

• Multi-factor authentication 
• Team management 
• Issue and request management (native or through 3^rd party integrations with tools like Jira) 
• Standardization template creation (policy, workflow approval, source code branch strategy, etc.)  
• Process standardization enforcement rules 
• Workflows and software development lifecycle management 
• Business requirement management 
• CI/CD pipeline configuration  
• Development project/product-specific technology selection 
• Maturity assessment survey configuration management 
• Activity and pipeline status audit log collection scoping and scheduling  
• Activity, pipeline and environment status and audit log collection, reporting and alerting 

Development Platform 

• Multiple tech-stack integrations (to facilitate platform orchestration). 
• Standard cloud infrastructure configurations (AWS, Azure, Google Cloud, Private Cloud) 
• Standard machine configurations (vCPU, RAM, Storage) 
• Standard tools and technologies selection and instance configurations (secrets management, IaC, agile planning and collaboration, document management, app dev tools, test automation, source code repo, CI/CD pipeline, deployment containers, security assessment, code quality, data tools, etc.) 
• Self-service development environment provisioning and automation 
• CI/CD pipeline automation (based on development project/product specific tech stack selected by dev teams) 
• Source code repo automation (based on development project/product specific tech stack selected by dev teams) 
• Containerized deployment orchestration (Kubernetes, Docker, etc.) 
• Deployment stage promotions (e.g. Dev to QA, to Staging, to Prod) 
• Security assessment and assurance 

Phew! You just realized that building your own IDP does provide you with some level of flexibility, but it begins to dawn on you how quickly this could become a slippery slope.   

You ask yourself the obvious question:

“Do I really need to build my own IDP?” 

Great question! 

The rationale for “buying” instead of “building” your own IDP 

Now that you have gathered enough data and realize how much you may be risking your job by blindly acting on building your own IDP, you make the wise decision to explore and consider other alternatives.   

You start by evaluating the factors that you would have to contend with if you were to embark on the journey of building your own IDP internally. 

• Time to delivery – the time it will take to deliver a fully functional platform with the minimum scope that will not only encourage adoption, but expand usage across the enterprise, could be unacceptably long.  This is especially true when standard development risks are factored in. 
• Development risks – challenges related to bringing together the appropriate resources, with the skillsets required to design, develop, build, deploy, maintain, and upgrade the platform cannot be underestimated, especially when one factors in the inevitable scope creep and quality concerns that usually arise. 
• Delivery costs – predicting software development cost is an art that is mastered by a few.  The risks of underestimating the overall cost of building and delivering a quality platform on time, and with the required scope to encourage adoption could be quite high.     
• Developer adoption and retention risks – software developers can be quite peculiar when it comes to adopting new tools they view as not adding any direct benefit to what they do – which is to build software. They will adopt any tool that improves their developer experience, and quickly reject those that do not.  Providing them with an IDP with an incomplete set of orchestration features is one way to lose them. This increases the burden of spending the extra time and money required to ensure the platform capability scope delivered meets the needs of the developers. 

• Integrations and add-ons – the enterprise and cloud technology ecosystem are growing and getting more complex with time.  This adds the extra burden on the internal IDP development team to keep up with the changes, further increasing the overall cost and development risks the team needs to contend with over time. 
• Support and maintenance costs – the internal IDP development team will need to absorb all operational costs related to support, enhancements, upgrades, deployment environment monitoring, and more. 
• Community – the added value of sharing/gaining knowledge from a broader developer community regarding the internally developed IDP will be limited to the open-source framework/tools adopted by the internal IDP development team. 
• Availability of commercial IDP alternatives – are there any alternative IDP Platform-as-a-Service (PaaS) solutions capable of delivering a more complete set of capabilities immediately?  
• Predictability of total cost of ownership and low, upfront costs – comparing potential IDP PaaS solution costs to the potentially high cost of internally building, supporting, enhancing, maintaining, upgrading and keeping up with integrations required to keep up with the ever-evolving technology ecosystem. 

You feel even more certain, after completing this evaluation process that you need to explore the “buy” option further. 

So, you set out on a discovery process. 

Why should you consider Calibo’s IDP? 

One of the first articles you come up with when you begin the research into IDP providers is a report produced by Gartner titled “Innovation Insight for Internal Developer Portals”, authored by Manjunath Bhat and Mark O’Neill.

As it says:

“Product teams often struggle due to disparate tools and disjointed workflows as they
accelerate digital transformation. Software engineering leaders leading platform
teams must establish internal, self-service developer portals to enable consistency
and scale cloud, agile and DevOps initiatives.” See image below.

The article lists Calibo as one of the commercial IDP platforms that should be considered as an alternative to Backstage.io.  This piques your interest, and you hop onto the Calibo website to check them out. 

Your deep dive into Calibo reveals that their platform already supports the majority of the IDP capabilities you’ve itemized, including the required developer portal and platform orchestration capabilities with 150+ integrations to top-tier technologies.   

You also discover that the Calibo platform supports additional capabilities that you hadn’t previously considered – a Data Fabric Studio that empowers data teams to unlock the power of data through intelligent data stack orchestration and automation.

The platform also includes embedded product release orchestration capabilities for product/project release managers to centralize the management and reporting of their agile teams and projects.   

“Great!”  You exclaim to yourself, acknowledging that this research is turning out to be one of the best moves you’ve made since you started this journey. 

But wait! You have one more question:

What if you want the flexibility to implement and extend the capabilities of the platform to address some unique requirements within the business?   

Upon further investigation, you discover that Calibo’s platform supports an API strategy that enables both the development of UI widgets and tech stack integration.   

You exhale and mutter a silent “Awesome, this will work!” to yourself.   

You decide to take the next natural step to request a demo from the website and take the evaluation to the next level.

Good move. You won’t be disappointed.

Want to know more about Calibo? Talk to us.