Using performance metrics when managing long-distance relationships
Go BackMastering the art of remote governance and agile cross-continental synergy cultivation to create a healthy relationship with your long-distance partners
Published on 19 June 2019
As a team, one of the biggest discoveries we continue to make over the years lies in our ability to consistently improve our working relationship with our overseas product development partner allowing us to optimise ongoing performance with our clients.
We have been lucky. Since our inception in 2015 we have changed partners only once (in 2016) and have since seen steady improvements in speed and quality from our Bangalore-based colleagues as a result of our complementing skill sets and our synergistic agile approach.
When managed effectively, such a relationship allows us increased confidence to make more daily product-related decisions and ultimately allows us to become rapidly efficient in giving our clients a taste of what they may be looking for by offering alternatives in proof-of-concept testing increasing their likelihood of making a long-term investment.
Consistency is king but equally important is being ready for the unexpected and the capacity to remain focused when challenged by connectivity issues, language barriers or when key overseas staff members require time to nurture a fever, bereave, or take a last minute day off to spend time with their children who are on their summer holiday.
Some time back I posed a ‘simple’ question to several discussion groups consisting of programmers, agile enthusiasts, project managers and product specialists regarding the best way for a small business like ours to manage the performance of a product development team from a long distance? Although the answers varied, to my great surprise, the majority of the responses from IT personnel stated that there is no magical KPI to track your product team if you want to be agile and offer quick solutions to your clients. Several experts even warned us that using KPIs of such will result in a negative impact on the developers by introducing the unnecessary stress associated with an approach that will most likely be viewed as micro-management in an environment that requires creativity, agility and that famous fail-fast logic necessary to sustain the innovation lifecycle.
As a producer of continuous improvement solutions, we are accustomed to challenging the status quo and being in disagreement with 'the experts'. In this case, wanting to try our own home-made KPI management tool, we paid little attention to those who advised us not to introduce performance metrics into our team of programmers and began testing a wide variety of KPIs seeking those which would increase efficiency while not sacrificing creativity.
Outsourcing product development tasks
is essential to maximise synergies
Over the course of a six month trial period using our KPI management solution together with our trusted Indian development partner, we have made several mini-discoveries regarding the best way to manage a long distant relationship of such. Our discoveries to date offer us a streamlined set of KPIs and managerial reporting supporting speed, agility, and creativity WITHOUT losing time in administration activities or getting bogged down in ‘analysis by paralysis’ challenges.
In doing so, I offer this post to anyone who seeks best practices in performance measurement when leveraging skills overseas to support the architecture, build and implementation of a cloud-based solution. In short, our findings to date tell us that being organised and disciplined doesn't necessarily mean that you will sacrifice creativity.
In closing, I am of the belief that the only way to truly understand what can go wrong in a business operating model of such is to see its dark side first hand, to take the time to capture the know-how acquired from the lessons learnt along the way, and to continuously acknowledge that there are very few experts in the field that have travelled on this same path and for that, use good judgement when 'asking directions from those who have never been to where you are going’.
Best of luck in your outsourcing endeavors and please continue to share with me any feedback, observations or best practices for the learning never stops, especially in this fascinating field.
Andrew Lenti
Chief Innovator at TOPP Tactical Intelligence
Our own home-grown tool first used
by us, now used in the market by many
Going global - Your KPI roadmap
The following outlines the 8 core metrics and the guiding principles of their usage that we have sucessfully implemented into our daily discipline to support the collaboration between our European-based office and our Bangalore-based product development partner.
Full details of our monthly management report (complete with statistics and graphical report)can be found in this SLIDESHARE. Furthermore, here is a 5-minute video of the KPI solution we jointly use to support our management reporting and day-to-day performance data collection activities.
Guiding principles
1. Performance of the overseas development team is assessed daily by TOPP TI (either at end of day for the current day or at the start of the following day for prior day’s performance)
2. In situations where there are no performance issues indicated, the KPI capture exercise should take no more than 3 minutes per day
3.In situations of poor performance ratings, TOPP TI will report to the Development Partner on a weekly basis to the Development Partner CRM highlighting issues encountered and their root causes. In instances where chronic root causes are present and performance is consistently trending downward, a discussion / review with the Development Partner CRM will take place
4.A final monthly performance rating will be given considering all daily KPIs captured during the course of the month
In cases of substandard final monthly performance rating (poor or very poor)
1. A meeting will held between TOPP TI and the Development Partner CRM to review and agree the final score
2. TOPP TI will be awarded service credits in months where the Development Partner’s performance rating is ‘poor’ or ‘very poor’
Quick and easy! 8 key daily metrics to optimise performance
As mentioned in the guiding principles above, this exercise will only be successful if the daily discipline is adhered to. In other words, if you get lazy and stop tracking when things are going good, you will have a poor data set to reference when you need it to manage periods of turbulence. Also, it is crucial that the key information required is quick and easy to identify and capture. If there are no issues to report, it should not take more than a few minutes to capture daily information on a small team.
To support this requirement we capture the following 8 pieces of information on a daily basis (usually in less than 3 minutes):
The basics
1. Product downtime:the number of hours our product was not available to our clients. This number ideally should be ZERO. No client wants to pay for a cloud service and be locked out. This number will be impacted by system outages, scheduled product maintenance and critical bugs released accidentally into the production area impeding your client from using their product. Industry standards for uptime % rates on cloud products should start at 99.5%. The development partner should have a contingency plan for remediation on the weekends and non-business hours and have a detailed rate card offering service credits when this figure dips below its required threshold.
2. Number of client milestone deliveries missed:This number is critical to your product delivery program and perhaps the most difficult to manage because it requires soft skills (e.g. negotiation & leadership) and is opinion driven. In short, in situations where we are preparing for a client product delivery or a demo with a prospect where our product is still not complete, we find it valuable to take the time to highlight to the product team the list of deliverables that are expected for the delivery date. This becomes your tactical ‘do-or-die’ action plan which your product team needs to put at centre stage. The ideal value for this KPI is always zero and will give a count for each agreed deliverable that was not ready for the agreed delivery date. Be cautious however for in agile environments it happens occasionally that new requirements are introduced to the project plan after the milestones are already agreed often impacting delivery. It is important to consider such activity and show empathy prior to penalising your team for missing a deadline.
3. Number of unplanned absences:Represents the number of days which a programmer is not present on a scheduled workday due to a last minute external factors (e.g. illness, family issues, etc.)
4. Number of untimely requests for absence:Represents any request for leave within the agreed notice period. On our team, requests for absences need to be given 30 days in advance.
Daily individual performance KPIs
1. Number of hours worked:This is one of the most critical numbers to establish with your programmers. It is collected daily but tracked collectively on a monthly basis. In our situation, we are entitled to 160 hours of product dedication from each of our full time employees. We recommend you establish a solid communication standard in tracking this number across the entire team. We do this very easily by asking each team member to send a ‘good morning’ message in Skype (or e-mail) on arrival and a ‘Leaving now’ message when closing the day. In addition to this we ask the same communication during lunch breaks. If necessary, you can become even more precise by tracking additional pauses throughout the day. In this case you start entering into the ‘micro manager’ space which is not highly advised when not deemed necessary. As mentioned earlier, we have been lucky with the maturity level of our team and have not required management at this level.
2. Number of unplanned schedule changes:Number of changes to the agreed work shift made within 24 hours of the work shift which cause disruption to the schedules of other team members. The ability to test and validate output as it is being delivered is fundamental to the quality of the final product. The more iterations, the better. As many programmers are accustomed to managing flexible work schedules to manage not only their personal lives but also the off hour requests that clients can often impose, it is critical to keep the schedules of the testers as synchronised as possible with the programmers to avoid downtime and waiting as work is passed from one to the other. By flagging each time the modification to a work shift had a negative impact on the other supporting team members, the programmer will become more aware of the importance to stay as in-sync as possible with his support team.
3. Number of hours 'shuffled' due to unplanned schedule changes: In instances where an unplanned schedule change causes confusion to the grand scheme of things in the daily collaboration, it is also important to note how many hours were re-allocated to better gauge the impact of the schedule change. If a programmer decides to start his day two hours later than usual causing 2 hours of downtime to your tester, this can prove to be extremely costly not only to the speed of product delivery but also to the level of quality for you lose testing iterations. Time lost due to this type of occurrence should be kept under watch, discussed, and re-visited. (to note: programmers while getting lost in concentration often don't see (or feel) this issue as much as the product testers do. The more visibility you can give to this issue and increased awareness, the better you off the team will be)
Daily performance rating:Describes the programmers performance based on a 1 to 5 scale with 1 being very poor and 5 being excellent. This is closing of the daily exercise. This is a subjective KPI for which its grading logic needs to be agreed between you and the Development Partner. It encompasses all 3 daily performance KPIs (5,6 &7) and further considers quality and timeliness factors such as chronic defects, connectivity blunders, as well as any confusion and miscommunication during the course of the day resulting from lack of concentration which caused slowdown to productivity. It is wise to maintain a living list a list of examples (root causes) that would justify a substandard daily performance rating to allow your product team to be more familiar with what types of situations are best avoided.
The monthly rating
1. Monthly performance rating: This is the final KPI assigned at the end of the month taking a cumulative view of KPIs 1-7 and the average score of KPI 8 for the month. It will be located on page one of your monthly management report delivered to the business lead at your development partner and will determine what level of collective review as well as remediation activity tu pursue in the upcoming weeks. In cases of poor performance, this indicator will also determine what level of reimbursement you will request from your partner due to the negative impact of substandard service received.
Utilising the above listed methodology together with our home-grown KPI management online solution, we are able to produce and deliver this Monthly Management Report in under an hour.
In closing
Going into our 4th year of long distance service relationship management, we are delighted to see progress in our efficiency in managing our daily production line but are quite aware the continuous improvement process never ends.
Since this 'study' was first published, we have been delighted to receive a great deal of feedback from both experts using outsourced business models, as well as service providers themselves.
From whichever side you are on, this is a fascinating field to be in as we watch globalisation test our great, synergistic potential as human beings and see its notable positive effect in the ongoing fight to end 3rd world poverty.
We are delighted to hear more about this so please contact us at info@toppti.com or leave comments sharing your experiences and feedback.
About the author
Andrew Lenti has been working with multinational organisations involved in business transformation initiatives since 1999. In that time he has been based in 6 different European countries as well as having two years of experience working in client service operations in the United States. He has rich experiences working with business leaders at all levels of the organisational hierarchy and has spent vast amounts of time working with shared services and outsourcing centres of excellences on business restructuring implementation projects. He is one of the founders of TOPP Tactical Intelligence Ltd, a European operational excellence software provider and is one of the original architects of PRESTO Digital Enterprise, the all-in-one continuous improvement business management system currently being used on 3 continents by large and small organisations as a tool in performance governance for quality assurance and operational restructuring. Visit www.toppti.com to learn more about PRESTO or contact Andrew directly on LinkedIn.
