Overseeing 30 municipal enterprises, two subsidiary public utility corporations (Transport Authority and Water & Sanitation Authority) and with 87,000 employees, IMM is responsible for a wide variety of sectors – including transport, environment, community services, health and wellness, vocational education, culture, and tourism.
The daily data produced by the organization, its subsidiaries, and the residents of Istanbul creates a vast data pool. Collecting all its data in one place would give decision-makers access to analytics about a citizen's transportation, water and energy consumption habits, and their socio-economic conditions, in a much shorter time. In this way, IMM could analyze citizen needs, for example, to identify residents eligible for social assistance.
Recognizing the benefits of being better able to effectively access, manage and process this data would have a significant impact on the lives of its citizens, IMM set out to rebuild its legacy systems and establish a big data platform.
Before working with Cloudera, IMM experienced challenges with data sharing and access between departments and subsidiaries. Due to national legislation and regulation and a lack of technical infrastructure, big data and artificial intelligence (AI) technologies could not be utilized by the organization for data-based management and decision-making processes.
IMM turned to Cloudera to help it process the structured, semi-structured, and unstructured data produced by the different applications, services, and systems within its large data pool. It implemented the Cloudera platform, Vertica Analytics, Kafka, Spark, the CKAN data management platform and Tableau’s data visualization tool. Analyzing the data using AI and machine learning (ML), it can now obtain new insights and apply learnings to improve and produce new services – bringing a necessary smart city data infrastructure to the municipality.
Creating a big data pool has edged the organization closer to achieving a panoramic view of its residents – with 800 data sets collected from 30 different sources and an average of five new sets being added regularly. Demographic and service usage data relating to citizens provide an end-to-end view and the opportunity to perform advanced analytics. Furthermore, with improved efficiency in data sharing – via the big data system and open data portal being connected through automatic processes – the data sets collected are also open to academia, students, start-ups, and to the public all around the world following the personal data protection authority regulations (KVKK).
Using its new big data insights, IMM set out to implement its Transportation and Traffic Excellence program – a comprehensive initiative designed to introduce advanced data science, AI, and big data technologies to modernize traffic management and planning. This new digital platform uses Kafka for streaming and is revolutionizing how the organization assesses traffic conditions in real-time, allowing IMM to anticipate congestion trouble spots, intelligently route traffic, and guide citizens on their journeys.
Supported by a United States Trade and Development Agency (USTDA) grant, the project is built on the organization’s big data architecture and has rapidly reached its short-term goals, placing it ahead of the target deadlines set in the national plan. Transportation is Istanbul’s biggest challenge with 50% of incoming requests and complaints received by IMM being transport-related, so improvements in this area, even seemingly small ones, are important. In this instance, quick gains were made in traffic forecasting, parking management, route optimizations, fleet management, and public transportation demand forecasting and pricing.
Now, instead of preparing city and transportation master plans with static data every five years, it can exercise a dynamic approach.
Additionally, against the backdrop of the pandemic, IMM utilized big data for crisis management. All action plans, from transportation planning to social assistance, were prepared using big data and analysis of the spread of the virus. This influenced the day-to-day decisions around emergency social assistance activities and public transportation.
With nearly one million people applying for social assistance from the municipality as a result of the pandemic, big data made it possible to evaluate applications and identify who to prioritize through field studies – including the elderly, people living alone, and those in need of care.