Toraksero.gr, an innovative online platform aiming to address the challenge of accessing concise, accurate information on political parties

In response to the wealth of data and biased sources available to the voters, helvia.ai developed a platform that incorporates a digital assistant powered by Large Language Models (LLMs), to act as an online debate. The platform is tailored to answer voters' questions and help them make well-informed decisions about the political party that most closely aligns with their values.

By leveraging artificial intelligence and natural language processing, the platform ensures an interactive and convenient way for users to research and compare the positions of political parties in Greece and the Netherlands. By providing brief answers derived from official programmes and accompanied by reliable sources, the platform acts as a trusted digital assistant. The open access system supports English, Greek and Dutch and has been deployed on two websites:

  • Toraksero.gr for the recent Greek elections (June 2023), which attracted around 30 thousand user sessions
  • nuweetikhet.nl for the Dutch elections (November 2023).

The importance of retrieving political information

People's abstaining from elections is a critical and constantly growing issue in democratic societies around the world. A significant proportion of the population avoids participating in elections due to a lack of information about political parties. The absence of a comparable tool for accessing and searching political programmes underlines the importance of retrieving political information. This is particularly true for the youth, who are unfamiliar with the political landscape, and for the busy working class, who do not have the time to thoroughly evaluate and make informed electoral choices.

Access to accurate and comprehensive political information is vital for voters to make properly informed decisions. However, information overload, biased sources and time constraints pose significant challenges for citizens seeking reliable political information. Participating in political debates and controversies often requires clarifying differing views and identifying information that is presided over and supported by official sources. This process can be time-consuming and overwhelming, especially for individuals with multiple responsibilities and limited time to research each political party's policy agendas.

As a result, many citizens may find it difficult to navigate and understand lengthy and complex political agendas, preventing their ability to form educated opinions and make reasoned voting decisions.

Empowering voters with the use of AI

Toraksero.gr was launched in June 2023 as a platform designed to simplify the search for political information and help citizens better understand political positions. The platform was created to address the time constraints and information overload that many citizens face when trying to navigate political agendas on topics like taxation, healthcare, and education. To solve this problem, toraksero.gr's expert team curated political agendas from various parties and made them easily accessible to users. Using artificial intelligence, specifically Chat-GPT, the site became a reliable resource that provided both short and detailed answers to users' queries. This saved time for busy citizens while also ensuring they had access to trustworthy information, allowing them to make well-informed decisions about their voting preferences.

The platform allows users to browse the political parties they are interested in and delve into their agendas. Using innovative artificial intelligence, the platform extracts relevant information from these programmes and generates answers to users' queries using exact words from the relevant parts of the programme. The platform promotes impartiality by providing sources of information, enabling users to access more relevant material directly from the official program. This interactive and transparent approach enables users to verify the answers provided and develop a comprehensive understanding of each party's position on specific issues.

The operational process

The platform's capabilities are based on an NLP system architecture that leverages the power of OpenAI's GPT-4. This architecture incorporates Retrieval-Augmented Generation with RAG to efficiently integrate custom data into large language models (LLMs) and build user confidence. Utilizing advanced natural language processing and machine learning tools, the platform has been meticulously built to understand user queries and provide accurate and relevant answers. During the development process, the AI model was trained on an extensive database of political data, policies and positions from various political parties, allowing it to learn patterns and connections within the data and produce informative, contextually relevant answers.

Considering the need for a wider application and to reach a wider audience beyond the Greek citizens, we decided to extend the support of the NLP system to the Dutch general elections scheduled for 23 November 2023. While maintaining the integrity of the presented NLP architecture, we hired another domain expert to curate the political agendas of the Dutch political parties. The new Dutch corpus includes 18 parties and was seamlessly integrated into a "Dutch" iteration of our existing architecture.

After a rebranding process to adapt the website content for navigation in Dutch (as well as English), we launched the system on the new website domain https://nuweetikhet.nl/ask?lang=en, which translates as "Now I know! 

Evaluating Toraksero.gr: metrics and user satisfaction

To assess the effectiveness of the Toraksero.gr tool, we used a combination of different methods to gather feedback, evaluate its success, accuracy, latency and design.

Initially, the toraksero.gr team conducted internal tests before the public release. Employees tested the site, asked questions and evaluated the accuracy and usability of the platform. Their feedback and observations played a key role in identifying potential issues or areas for improvement. Also we sought feedback from external individuals, including non-experts. Each user was given a unique URL and a specific access token, allowing up to specific access token with a limited number of requests. We received feedback from alpha and beta testers, which helped us identify bugs, etc.This process facilitated the identification of small bugs, refinement of the visual design of the system, and improved the  performance in total. Next, our team employed Facebook and Google ads to reach a wide audience and encourage user engagement with the tool. This approach facilitated interaction with the site and gathered valuable feedback from a wide range of users.

Thirdly, we integrated a 'contact us' section with a chatbot into the platform and encouraged users to provide feedback directly through this chat. This allowed for real-time communication with users and simplified the collection of feedback on the tool's performance, user experience and accuracy of information.

The team, last but not least,  used social media channels such as Facebook and LinkedIn to post about the site and invite users to share their thoughts. Comments and interactions on these platforms provided additional feedback on the tool's usability.

Now let's look at a more detailed evaluation of the platform:

  • User Satisfaction: Comments collected through automated chat, comments from social media posts and direct user interactions were analyzed to assess user satisfaction with the performance of the platform and the quality of information provided. 

Here are some authentic and honest comments from users who have experienced the transforming power of toraksero.gr:

  1. ‘’Congratulations, finally an official information source for citizens to vote consciously and responsibly.’’🙌
  2. ‘’Truly wonderful work and extremely useful for citizens who want to be informed without having to search. Is this the end of (non-subjective and guided, simply reproducing statements without critically examining them) journalism? Hopefully, it is.’’ 🧠
  3. ‘’Congrats guys ! What hit me is the following : Until now, political parties and their advisors had to invent new ways to impress and engage the voters. Billions are being spent globally on that! The media and the voters were all together intertwined in a very close relationship- who’s to win, who's to blame etc. Now, you’re bringing the new generation of critics : AI, so parties have to win the LLMs too.’’😃😃
  4. ‘’Just wanted to take a moment to thank you for developing this utility’’ 👏

  • Accuracy of answers: The ability of the tool to provide accurate and relevant answers to users' questions was also assessed. The answers were compared with the official source and assessed for consistency with the political programmes of the parties concerned.
  • User engagement: Analyzing statistics such as the number of user queries and frequency of interaction allowed us to measure user engagement with the platform. This analysis provided valuable information on the effectiveness of the platform in terms of attracting and retaining users. Exceeding expectations, Toraksero received over 29,629 queries based on internal statistics, showing significant user interest. This engagement highlights the value of the platform for individuals seeking updates on the political landscape. 

Following the launch of the platform, we used "like/dislike" buttons to collect user feedback, receiving a remarkable 74% of positive comments (likes) and indicating high user satisfaction. Of the total queries, only 198 (4.4%) revealed instances in which LLM provided an incorrect answer. These included cases where GPT-4 answered incorrectly (selecting the wrong source document) or claimed not to know the answer, despite retrieving the correct source document via semantic search."

  • Feedback: User feedback and recommendations, both through automated chat and social media comments, were reviewed to identify areas for improvement and gather recommendations to improve the tool's functionality and user experience.

Limitations

While toraksero.gr has proven to be extremely useful, certain limitations should be noted. Foremost among these concerns is the content curation process, which depends on the availability and accuracy of information provided by political parties. Gathering up-to-date and consistent information from all parties is a challenge due to different inputs, communication strategies and limited resources for distributing information. 

Since Natural Language Processing (NLP) systems are based on textual data, we faced limitations in including political parties without publicly available agendas. For the Greek elections, we excluded four smaller, newly formed political parties that had not uploaded content. However, we are committed to incorporating their agendas into the NLP system to enhance public awareness in future elections.

Conclusion and next steps 

In conclusion, toraksero.gr has proven to be an extremely valuable tool for voters, facilitating the discovery of political information and aiding in making well-informed decisions. Users have provided positive feedback ,along with recognition from the media, highlighting the significance of such tools in ensuring citizens access to information regarding politics. 

Toraksero.gr, as a generative AI tool, serves as an ideal model for developing similar platforms in various domains, bridging information gaps and empowering users in different fields of knowledge. Through utilizing conversational AI, we can foster a more informed and engaged society. 

For inquiries please get in touch at contact@helvia.ai.