NSoft’s Data Analytics TeamStories | 08 May 2019
Introducing NSoft’s BAS and NDS
The formation of BAS (Business Analytics Specialists) team started 2 years ago with the arrival of a small BI team to NSoft in order to face new challenges in a completely new industry.
Considering our business background and experience in Data warehouse (DWH) development and maintenance, reporting and data provisioning, as well as the previous, almost decade long, cooperation and teamwork, we were always well aware of the importance of data and its impact on business performance. But what we have encountered at NSoft has solidified our thinking that the sole data provisioning is the thing of the past – the present data provisioning is being performed by automated systems without human interaction. That was the starting point of something that consequently evolved into a completely internally developed full-fledged Data Analytics Solution backed by BAS team and data engineers in NDS (NSoft Data Service) team. Implemented, with the best practices in mind, we are currently operating Data warehouse (DWH) capable of nearly real-time processing and ingesting millions of data points per second which then feed into our analytics layer. It resulted with a clear new goal to strive for – Data Analytics.
What is a Data Analyst?
Data Analyst by definition is someone who is able to successfully translate large quantities and a variety of data into plain language understandable to others. But the role of the data analyst isn’t strictly tied to data “translation” and understanding. It’s more of a role that incorporates generating clear and advisory actions that have an immediate business impact. Within the context of NSoft, upon initial observation of the entire business process, we have soon realized the requirements for a data analyst and how one could contribute. This included familiarization with vast quantities of data, from numerous and various sources, combining, preparing and finally analyzing and extracting the value. What seems like a list of cursory terms is actually enormous effort that involves preparation, analysis, development, and usage of a system that enables BAS to govern valuable data and generate value for NSoft and it’s partners alike.
Importance of Turnkey
Striving towards common growth is the simple definition of what the Turnkey is. Coming to NSoft and adopting the Turnkey concept has made us completely change our mindset regarding the usage of product and player generated data. For NSoft the turnkey product solutions represent symbiosis (through a revenue share model) with our partners with the common goal of mutual growth. Taking it all into consideration, through time, we have realized our mission statement has shifted: “We will not be providing data, we will provide the value!”
Data Analytics as a product aid
One of the ways through which data analytics provides value is by improving the existing product portfolio wherever possible. By providing services of statistical analysis, testing product logic, identifying malicious player behavior as well as positive player feedback, we aim to implement our findings and insights into bettering each of our products.
Based on our analyses and observations, the numerous improvements, tweaks, upgrades have been made to our virtual games (Lucky Six, Next Six, Greyhound Races and many others) which have led to greater recognizability of our products and far better player retention.
NSoft’s Lucky Six is a market leader in Virtual Games and successfully competes on a global scale market, so one of our goals is to track player trends and keep the game fresh and interesting for our current players, as well as appealing and exciting for the new ones.
In Sportsbook segment, for Pre-Match and Self-Managed, even though it is not critical in terms of speed, extensive analytics of past period including relevant predictive measures are crucial for bookies’ micro and macro decision making.
Near real-time In-play tracking and alarming that we provide is extremely useful for making snap decisions regarding certain matches and offers that could negatively impact our partners and us in the process.
Other products from NSoft’s rich portfolio receive the same treatment from our Analytics team in order to provide the best possible product to our partners and the best gaming experience to our players.
All of that combined sums to a sentence that best describes our aspirations:
- Through data analysis, we aim to improve existing products and help to create new ones, to provide our partners with the better business results, give the players the best possible playing experience and maintain the highest standards our company has set up and insists on.
Data Analytics as a partner aid
Everything previously mentioned has resulted in the development of workflows capable of consuming and digesting this enormous amount of information which enables us to provide our partners with momentary information regarding anything of interest. We take all the relevant gaming Key Performance Indicators (KPI) into consideration when aiding our partners’ decision-making process.
Through our Average Revenue per User (ARPU), Acquisition Rate, Retention Rate, and Churn Rate reports our partners receive direct insight into players’ behavior, which helps them immensely in bringing new players, as well as retaining the existing ones. By identifying the reasons for player churn we can provide our partners with actionable insights. Actionable insight is more than an information with context, it’s a prescription for a clear course of action. The final goal of those actions is an increase in partner’s revenue, as well as an increase in confidence in our information provisioning.
Also, for improving their overall business performance, we provide reports like Gross Gaming Revenue (GGR) and Net Gaming Revenue (NGR). A macro overview of partner’s business is an essential tool for future strategic decision making.
Our team has streamlined dashboard generation into an assortment of standardized report packages offered to all of our partners and in-house teams and individuals alike. We also react to partner’s needs for any information regarding the usage of our products and their immediate business impact. Regular reviews of those requests (also occasional re-reviews) lead to new reports being added to standardized packages for all our partners, resulting with an increased portfolio of NSoft offering. That workflow enables NSoft to identify the partner’s needs before they occur generating bilateral value.
The technology that made it all possible
The technology stack involved was designed to be robust, easily scalable, adaptive and highly available and it required an enormous effort.
In our DWH implementation, we’ve chosen a streaming solution instead of ETL processes. As a starting point we should mention in-house operational data stores used for streaming sources:
Currently, from NoSQL databases, we have MongoDB as a source, and from SQL databases we’re currently using MySQL and PostgreSQL. Additionally, due to our partners’ specifics and requirements we are supporting various CSV, Google Sheets, SQLite, MS Excel data enrichments which can be blended at source or anywhere in the streaming pipeline seamlessly.
For data streaming Apache Kafka is used as a backbone and it’s accompanied by ZooKeeper, Kafka Connect, Kafka Streams, Kafka Schema Registry (as we’re using AVRO for primary streaming format) and Confluent kSQL.
Through streaming, pipeline data is enriched and transformed to ensure resulting data form compliance and quality and minimize data cleansing overhead.
Due to high-level data processing in the pipeline we’ve also decided to implement a caching layer using Redis.
For resulting DWH solution we’ve opted for Vertica, a leading Enterprise SQL Column Store Database solution, as our long-term store and analytics back-end, as well as opting for Elasticsearch for our near real-time data insights.
The Tableau is our primary information presentation driver, coupled with Superset, Metabase and Google Data Studio. Tableau was versatile enough to meet all the requirements we have had from an information distribution platform. Grafana and Kibana were our primary choices for visualization of near real-time data.
Programming languages in use are Java as a primary, Python as a secondary but as it usually happens, you need to expand your palette to deliver a good product so Bash, NodeJS, and PHP happen to be regular visitors of our team.
Monitoring needs are being effectively covered with Sensu for the systems part and Prometheus for service monitoring – ranging from availability checks to performance metrics.
Most of our stack is powered by Docker and due to stack size it was impossible to keep things smooth without orchestration solution, that’s where we decided to use Kubernetes with Rancher on-top for easier management.
Current CI/CD pipeline is custom implemented using Helm based Jenkins and Kubernetes publishing. It’s also worth mentioning that our K8S setup is expanded with Istio service mesh, MetalLB, fluentd, Rook and many other improvements compared to the standard issue of Kubernetes setup.
The entire setup is currently configured on bare-metal dedicated server machines and is horizontally and vertically scalable, as well as highly-available. The configuration of such stack led to high proficiency in DevOps department and faced us with some interesting challenges.
EGR Global has announced the official shortlist for this year´s EGR B2B Awards. NSoft has been listed for the best betting company in three categories: Bingo Supplier, Data analytics Partner, and Sports Betting Supplier. Being shortlisted in three different categories demonstrates not only the validity but also the level of excellence that NSoft provides including the acknowledgment to the development direction that the company has undertaken.