We use cookies to collect and share information. Read our privacy policy to learn more. You consent to our cookies usage if you continue to use this site.
Case study

Cleaning Up Big Data

Industry: Banking and Finance Region: Western Europe Technology: Java Volume: 0.5 man years
The Challenge
Big data comes with big promises. When used to support decision making, big data creates a realistic picture of the market, annihilates human biases, and instills true confidence. But when big data becomes a swamp, its value decreases to zero.
Our Mission
Gathering data was never a problem for one of the largest European banks, but they were unable to explore, understand, and use it. Drawing on our knowledge of algorithm development and data mining, Softaria set about developing a tool to modify the data so that it could be properly used.
The Solution
We created a solution that brings an existing database and new entries to a unified format. At the core of the solution is a scalable clustering algorithm paired with a complex analysis algorithm. When the tool was finally deployed, 10 million entries containing duplicates were converted into a clean and usable depositary of customer data.
Robust Scientific Approach for Impeccable Data Quality
What we did:
Developed advanced algorithms for data analysis and splitting the data into clusters.
Minimized time needed for optimal algorithm performance considering database inactivity requirements imposed by the business.
Conducted a series of continued tests on provided datasets for each algorithm modification for evaluation purposes.
Managed a complex interactive algorithm development and testing process consisting of many prolonged iterations.
Enabled the client to clean up a massive database of 10 million entries, remove duplicates, and fix data inaccuracies.
Equipped the business with a long-term data cleaning solution to process existing and new entries.
Want to know more about the project?

Read more case studies

Sorry, your files couldn't be uploaded. The upload mustn't exceed 10mb.
No file chosen
X
Thanks for contacting us.
We'll review and get back to you shortly.