Our impact Dashboard Impact Dashboard FAQ Street League is committed to transparency and developed this online tool so you can see exactly how we’re doing throughout the year. It is structured alongside a high level overview of our theory of change (how Street League expects outcomes and social impact to be achieved, given the work we do). You can see how many young people we have been able to support, as well as those we haven’t been able to help and why that is. We put this alongside national data, which gives further insight to the need for Street League's programmes. Why did you decide to develop an online dashboard? Street League is committed to transparent impact reporting, and sharing our data publicly, in a real-time dashboard, was an efficient way to move this forward. In 2016 we shared our “Three Golden Rules” of reporting and as explained in our 2017-18 annual report, we hoped a dashboard would move that conversation forward. You are now looking at 'version 2.0', launched in October 2018 with new filtering and features introduced to reflect the feedback since we launched the first one in November 2017. We want the dashboard to bring our Three Golden Rules to life and it starts with an overview of our Theory of Change - how we expect outcomes and impact to be achieved through the work that we do. It roots this in the social need we want to address - youth unemployment. It then sets out the young people who start on our programmes, those who disengage and the reasons why, young people's progressions and finally sustainment of jobs for at least six months. We only use full datasets, include the absolute values, and every progression and sustainment reported on has externally verified evidence to back it up. It is common to have to wait a year or more to understand what a charity’s impact has been, but we know that in order to learn from our data and improve our programmes, we need to respond much more quickly than this. Internally, we have been using detailed monthly scorecards and dashboards to understand how well we’re doing since 2015, and sharing this publicly seemed a great next step. We know that there are other social impact dashboards out there, but we think the real-time nature of this data being pulled through directly from our database is a true step forward in transparency – none of the data is edited and if the results aren’t good, that’s what it will show. How do you think it changes your relationship with donors? As well as seeing how we’re doing month-to-month, the dashboard allows you to interact with and filter with the data so you can ask your own questions and see how well we engage and progress different young people. We have to limit the granularity of filtering though, to protect young people's data - this was something we learned early after launching the first dashboard. We have had a fantastic response overall from funders and partners with many providing input to make improvements. We have also had 134 people complete the feedback form on the webpage. 54% (73/134) strongly agreed with the statement 'This dashboard has helped me to better understand Street League's social impact', whilst 19% (26/13) strongly disagreed. Their suggestions as to how this might be addressed informed version 2.0. What other benefits do you think it will bring?Hopefully, it will spark a wider conversation in the sector – for charities and funders. We feel that it’s time that the charity sector moves away from an over-reliance on stories and also has the hard data to back this up. We hope it will encourage even more organisations to publicly endorse our three golden rules and put them into practice. This honesty should help the charity sector communicate as a whole to funders and partners that what we do is hard work and to help the people who really need our support means we need to take risks and sometimes we’ll get it wrong. How do I understand trends and know whether or not you're improving?As of September 2018 (the final issue of version 1.1) we save a PDF version of each month's dashboard that you can download and view. We include them as PDF files (and not interactive Power BI files) to facilitate automation of the upload process, minimise risk to individual's data, and minimise the data required to access the webpage. What changes were made between versions 1 and 2? We made several small changes to the dashboard since it was first launched in November 2017 and were on version 1.4 by the time we launched our most recent upgrade in October 2018 (version 2.0). These included aggregating postcodes to local authority level, changing terminology from 'transgender' to 'other' as we expanded the list of choices, and added some further clarifications when questions were posed. Changes for version 2.0 include: Updated visual design as Power BI has enhanced the opportunity for us to add images and backgrounds Alignment of the dashboard tiles with a high-level overview of our 'theory of change' - this could also be thought of as the journey/process a young person takes at Street League. We hope that this gives greater insight to Street League's impact. Overview of the proportion of young people from the most deprived communities (as per the indices of multiple deprivation) who start our programmes, progress into an outcome and sustain for at least six months. We feel this best summarises our impact. Addition of external datasets to give some context to Street League's efforts to end youth unemployment Added filters for age ranges and ethnicities (more detail on that below) Removed 'other' as a filtering option for genders as it was raised as a concern that this was not inclusive terminology for young people who do not self-identify as male/female. Who do you include in White / BAME categories? White: White-British, White-English, White-Irish, White-Scottish, White-Welsh BAME: African/African British, Any Other African Background, Asian/Asian British-Bangladeshi, Asian/Asian British-Chinese, Asian/Asian British-Indian, Asian/Asian British-Other, Asian/Asian Pakistani, Bangladeshi/Bangladeshi British, Black/Black British, Black/Black British-African, Black/Black British-Caribbean, Black/Black British-Other, Caribbean/Caribbean British, Chinese, Chinese/Chinese British, Gypsy/Irish Traveller, Indian/Indian British, Mixed White/Asian, Mixed White/Black African, Mixed White/Black Caribbean, Other, Pakistani/Pakistani British, White-Polish, White-Other What would you tell other charities who are thinking of doing something similar? We would encourage as many charities as possible to strive for the most transparent level of reporting. Whether that’s adopting all of the Three Golden Rules, focusing on getting evidence to support your social impact, sharing your challenges openly so others can learn from them, or providing more regular updates publicly on your social impact. One piece of advice would be to ensure that if you’re sharing real-time data updates publicly, you need to be confident of the data's quality, which is a much bigger organisational challenge than the technical aspect of creating an online dashboard. It has been a seven-year journey to get to this point for Street League, and there has been a lot of hard work to get our monitoring and evaluation up to a standard in which we have 100% confidence to share the data publicly. Key steps have been: Developing our Practice Framework, which sets out the key points along our programme and each young person’s journey, that gives a common language across all of our teams and programmes. Theory of change analysis with Impetus PEF to review and update what we measure and why Implementing a participant database so we can track every single young person’s journey from their starting points, and their skills development through the programmes, through to sustained six-month outcomes. This enables us to use full data-sets in all of our reporting with no extrapolation. Introducing a four-stage internal audit process to check the evidence behind all of our outcomes, for example a payslip from a young person who has moved into work. If an outcome doesn't meet the gold-standard we require it is returned to the team until they are able to do so. We record common issues to inform staff training and policy updates. Monthly reports to all teams that summarise their performance in detail and quarterly 'missing data reports' that ensure accuracy and completeness of our datasets. Data audits across the organisation on a quarterly basis that ensures the rigour of our data collection and development plans for every employee who inputs data on our database. Continuous review and training to make sure that data is collected in a consistent way across the organisation. The key thing is that this is a journey, which was led and championed by the Board, Chief Executive and senior management team. This leadership means that it is a priority for the organisation and is a key part of our organisation’s strategy. What’s next for Street League? We continue to work with partners across the sector to explore how we could increase alignment - in the data we collect and the principles that underpin its collection and reporting. This would facilitate benchmarking and we also hope that Government will support charities' efforts to be more transparent and efficient by granting access to national datasets and helping us understand our relative impact. We are also developing a predictive modelling tool that utilises our data to better anticipate the support needs of young people based on their starting points. Where does the data come from?This dashboard pulls internal data from our participant database, Hanlon. External data for unemployment rates is drawn from the Office of National Statistics' webpage. Our staff input data on every young person's journey. Hanlon is a monitoring and evaluation system which allows us to record where our young people are coming from (their highest educational attainment, their postcode mapped against the Indices of Multiple Deprivation data, their length of time unemployed and any barriers they were facing to getting a job like a drug use or housing issues). We also track their journey through our programme, the outcome they achieved and most importantly whether they stayed in the job for at least six months. We track when we aren’t able to help a young person, for example when they left our course early without achieving an outcome, and then feed all of this data back into re-designing our programmes. Most importantly we introduced a rigorous internal audit that required every outcome and sustained job (6 months) we achieve to be validated – for example, a job outcome is only valid once we had a photocopy of a first months payslip or a job offer letter. We need to prove what we do. How often is the data updated?We update our data on the 7th of each month, or the closest working day if it falls on a weekend or bank holiday. There is a countdown clock at the bottom of the page which shows when the next data will be updated. How did you make these dashboards?The dashboards are built using Microsoft's Power BI software, with data pulling in from our Hanlon participant database. We can then embed the custom dashboards into our existing website platform using HTML code. How much did this cost to produce?The additional financial cost to Street League of creating this real-time impact dashboard was £264 per year, which was for buying user licenses to use the Microsoft Power BI software. Existing costs, which we were already paying for, include having a participant database through Hanlon and a website platform through Raising IT. The dashboard was all created in-house with no external agency fees.