Why We Built Tasktop Data

Posted by

We have a detailed web page that explains what Tasktop Data is and some great videos that explains what Tasktop Data does.  I thought it would be interesting for our readers to get some visibility into what were some of the drivers behind why we created Tasktop Data.

Like many innovations, features, products, and integrations that we’ve created over the past 8 years, Tasktop Data came from talking to customers, prospects, partners, and our employees.  At Tasktop, we have a long history of eating our own dog food. Culturally, we have a high bar and demand a tremendous amount of ourselves and are often a great source of feedback for improvements to our products and documentation.


And like a lot of what Tasktop creates, the problems we so uniquely solve may not be all that sexy but we work hard to solve the most technically challenging software delivery pain points in elegant ways.  We’ve heard this story dozens of times and finally had the bandwidth to attempt to do something about it. The Tasktop Data story goes like this…

Customer X’s Program Management Office (PMO) has been using CA PPM (formerly Clarity) for upwards of 4 years for Project Portfolio Management.  Their testers/QA department has been using HP Quality Center since the Mercury days, have an active Quality Center of Excellence, and are located on a separate campus than the PMO albeit in the same city.  For the past 2 years, Customer X has had a litany of people (some dedicated and some come in as needed) to move data so they can produce, maintain and look at software development and delivery reports to see how the company’s software organization is performing.  The team had created ETLs (Extract, Transform and Loads) that flowed projects, defects, and requirements from CA PPM and HP Quality Center to a relational database.  Every time the PMO decided to upgrade to the latest and greatest version of CA PPM and every time the testing department decided to upgrade to the latest version of HP Quality Center (even if they were patches), the ETLs stopped working and they’d bring in a team to redo all of the ETLs.  If a new project came on board and wanted a different customization or required fields or workflow in HP QC, that typically broke their ETLs. The ETLs ran on massive batch jobs, which were a maintenance nightmare and worse yet, the batch jobs often brought the QC and CA PPM servers to their knees.  Lather, rinse and repeat with various other PPM, QA, and ALM tools. 


The Return on Investment (ROI) case for Tasktop Data to facilitate helping companies like these get the data into their relational databases is valuable enough to be driving the significant demand that we’ve been seeing.  Tasktop Data easily pays for itself in the savings from the resources used to create, maintain, and recreate those ETLs let alone the value of those redeployed resources on projects that can drive competitive differentiation.  When you add on employee turnover to the team managing the ETLs, the returns are much more dramatic.  However, the story actually doesn’t end there though it could.  


For example, Company X like many organizations is trying to differentiate on software and deliver that software faster as doing so has been shown to drive real competitive advantage.  As such, they are in the process of rolling out Agile methodologies and a key business unit has decided on Rally as their tool of choice, hoping to see the benefits that best in class software shops have been able to achieve.  Defect and user story data from a SaaS Rally workspace and 10 on-prem JIRA instances now needs to be added to the same relational database that contains their QC and PPM data.  And they just noticed that this is going to be rather tricky to do with their ETL team as that team does not have access to the Rally database that is SaaS. To complicate matters, there are dozens of different project schemas across each of the 10 JIRA instances.  They love Rally Insights, but those only show what’s natively in Rally and what Tasktop Sync synchronizes into Rally, missing many artifacts across the other tools.  So they are scratching their heads considering whether they need to go for headcount for Java developers to create a Rally API scraping utility and figure out how to get that data into the reporting database to correlate with the data locked in the other tools. Each tool is showing great metrics for its part of the lifecycle.  For example, Rally Insights is showing the progress and results of the Agile Transformation, while HP ALM/QC is showing the quality and performance metrics, and JIRA stats on team level issue happening in development.  


But nothing is showing the end-to-end flow.  And companies are desperate to get a handle on the end-to-end process of software development and delivery.  They tried and failed miserably with some vendor created integrations as well as services created integrations between HP QC and Rally, because they thought they could stand it up and maintain that on their own, but the integration lacked the robustness they were expecting and needing.  Examples of the missing robustness included getting log files, dealing with conflicts, scheduling, QC workflows, etc. As they looked at what drove competitive differentiation, they came to the conclusion that they’d rather buy a solution than staff for it.  With Tasktop Data, hooking up this morass of different lifecycle data sources to their existing data warehouse becomes trivial.  Now this organization can deploy the most modern of Agile tools, while meeting their governance and compliance requirements that are required by reporting tools connected to the data warehouse.  By applying the Tasktop approach of being tool agnostic, we have been able to solve one of the hardest problems facing organizations doing large-scale Agile and DevOps modernizations… providing a real-time and normalized source of lifecycle data to plug into the existing data warehouse and BI tools used for reporting.  And you should see what software lifecycle analytics engines such as HPs Predictive ALM, IBM’s Jazz Reporting Service, Rally Insights, etc. or Business Intelligence tools look like when Tasktop is pumping information and data to them.


As Tasktop Sync is being relied on in some of the biggest and most demanding software shops in the world, we kept hearing variants of this exact same story.  Interestingly, we would hear this story consistently about 12-15 months after the companies had Sync up and running.  More recently we’ve been hearing about this from our VPs and Directors of Software Delivery in first meetings we were having about their integration needs.  The name of the end points may be different than CA PPM, HP Quality Center and Rally, but we’ve heard similar stories from financial services companies, airlines, automotive companies, healthcare, manufacturing, medical devices, retail, etc.  Whether the goal is compliance and audit, improving processes, delivering software faster, ensuring code reds have the appropriate visibility, etc., we kept hearing the same desperate plea for help.  Tasktop Sync can feed your existing tools with the additional data needed to expand their scope of reporting.  And for reporting across all tools, Tasktop Data can be leveraged to connect to your data warehouse to your Agile/ALM/DevOps tools without having to spend even more person years trying to ETL each tool or create and re-create API bridges each time you upgrade or deploy new projects.   We are actively working on adding more integrations, data warehouse tools and partner analytics tools and methodologies. So please stay in touch!