Description
Samuel Spencer https://2016.pycon-au.org/schedule/135/view_talk Historical information on Australian Parliaments has been available in the Parliamentary Library's flagship publication the Parliamentary Handbook - an extensive almanac with biographies, tables and records dating back to Australia's federation. This data is used as a way to track key social issues, such as length of service, gender representation in parliament and historical election information in an authoritative format.
To improve access to this information the library began development of a mobile app which evolved into a complete Django-based data management system for managing parliamentary biographies and service histories based on Popolo, a civic data framework for the management and dissemination of parliamentary information. Along with interactive biographies and records of ministries and parties, the system for the first time allows users to build custom tables from complex queries that are dynamically updated as new information is made available.
Coupled with this is the development of a biographical data management system that will ensure that records of new parliamentarians and future changes to existing parliamentarians are captured in a single system.
In this presentation, we cover the challenges and successes in digitising over 100 years parliamentary data, including migration, data cleansing and data trust issues. We also provide a technical breakdown of the chosen framework and infrastructure, and issues during development especially when dealing with imprecise or incomplete historical records. Lastly, we cover the Django apps for data modelling and querying built for this project that have already been released as open-source.