Big Data Appliance (BDA) Cloud Migration

Background

Our client, an American payment processor operating nationwide, required assistance with migrating their data from the current Oracle Exa-data based platform called BDA to a cloud-based platform utilizing AWS and Snowflake technology stack. Their aim was to overcome the challenges associated with maintenance, scalability, and data availability while optimizing performance.

Current Challenges

Maintenance: The existing system, which was nearly 8 years old, presented difficulties in maintenance due to lack of support, high costs, and outdated connectivity with modern platforms.

Scalability: Different lines of business within the organization had varying volumes of data and distinct needs. The current infrastructure lacked the ability to scale up and down according to different scenarios on a daily, weekly, and monthly basis.

Data Availability & Performance: Reporting and consolidation processes were predominantly manual, consuming valuable time. This limited business users’ ability to access data for independent analysis.

Solution

Our proposed approach aimed to address the challenges mentioned above and provide an efficient cloud migration solution.

Raw Layer: Data from various sources was migrated to the Amazon S3 storage service using ETL (DataStage), where it would be stored in its raw form.

Curated Layer: Utilizing stored procedures and Snowflake tasks, data was transformed from the raw layer to the curated layer, where it could be organized into tables or views.

App Layer: Views were created in the app layer based on user requirements. These views could be utilized for data consumption by users, reporting, and integration with downstream systems.

The Results

The successful migration to a scalable and cloud-based platform using AWS and Snowflake technology stack brought several significant benefits to our client:

Efficiency and Scalability: The new cloud-based platform offered scalability, providing the flexibility to adjust resources based on varying data volumes and needs. This not only improved performance but also reduced maintenance efforts and costs associated with the outdated BDA platform.

Improved Data Availability and Performance: By automating the data transformation and consolidation processes, the client gained faster and more efficient access to data. This empowered business users with self-service capabilities for independent analysis, enhancing overall productivity.

Overall, the migration to a cloud-based platform equipped with AWS and Snowflake technology stack enabled our client, the American payment processor, to overcome their challenges, enhance scalability, optimize data processing, and improve business operations.