Contribute Media
A thank you to everyone who makes this possible: Read More

Building a RESTful real-time analytics system with Pyramid

Description

Andrii Chaichenko - Building a RESTful real-time analytics system with Pyramid [EuroPython 2015] [23 July 2015] [Bilbao, Euskadi, Spain]

CeleraOne tries to bring its vision to Big Data by developing a unique platform for real-time Big Data processing. The platform is capable of personalizing multi-channel user flows, right-in time targeting and analytics while seamlessly scaling to billions of page impression. It is currently tailored to the needs of content providers, but of course not limited to.

  • The platform’s architecture is based on four main layers:
  • Proxy/Distribution -- OpenResty/LUA for dynamic request forwarding
  • RESTful API -- several Python applications written using Pyramid web framework running under uWSGI server, which serve as an integration point for third party systems;
  • Analytics -- Python API for Big Data querying and distributed workers performing heavy data collection.
  • In-memory Engine -- CeleraOne’s NoSql database which provides both data storage and fast business logic.

In the talk I would like to give insights on how we use Python in the architecture, which tools and technologies were chosen, and share experiences deploying and running the system in production.

Details

Improve this page