Microservices and Serverless Architecture for computing streaming data from field devices

Serverless and Microservices are used together to create highly scalable backend solutions for some of the popular mobile and web platforms. 0ne of our UK customers wanted to build a fitness tracking app which is integrated with a wearable fitness band. For this product, a highly scalable backend is required which is capable of processing high volumes of health data coming from millions of users, has the capability to provide AI based gamification in the form of targets, challenges and goals over fitness activities for user engagement. Blockchain based rewards implementation which are used for purchasing products available on the app E-commerce store. STANRA designed and built the application. 

Challenges:

  • Regular Syncing of sensor data from devices
  • Real time data aggregation
  • Real time analytics and gamification 
  • Scalable to process data from millions of devices

The Solution:

Stanra team implemented following solution:

  • Serverless deployment as there are a lot of microservices and each of them should be highly scalable and available. Serverless using AWS Lambda takes care of scalability and availability of services
  • Built microservices using Spring Boot for the mobile app to read processed health and user data, sync health data to the backend for computations, deployed them on Lambda and exposed them using API Gateway for scalable and faster response to the mobile app
  • Built backend services using spring boot for all complex backend processing logic, AI logic which are deployed on Lambda and triggered either through SQS on write or through Cloudwatch events thereby allowing asynchronous processing
  • DynamoDB is chosen as a database for all User Data, Logging unprocessed health data
  • Neo4j is chosen as a graph database for storing aggregating data as it maintains chain and has a faster search capabilities
  • SQS was chosen as a queue for all asynchronous processing through automatic triggering of AWS Lambda

The diagram below explains the solution designed by STANRA and implemented using Spring Boot microservices and backend services:

The Takeaways:

  • Wherever possible opportunities must be explored for asynchronous processing of compute intensive data. In most cases it will provide much higher  performance and user experience 
  • Asynchronous processing of data along with use of Neo4J for storing processed data enabled Real time analytics for Health Data
  • Choice of Serverless along with microservices provided required hardware level scalability and availability of the server application
  • Choice of ReactNative enabled simultaneous release of Android and iOS versions of the application eliminating the inevitable time lag that would have come if the two applications were developed in parallel
  • Automation of complete release cycle enable efficient testing and UAT deployment on daily basis