We provide IT Staff Augmentation Services!

Big Data Engineer Resume

4.00/5 (Submit Your Rating)

Plano, TX

SUMMARY:

  • 10+ years of experience involving Python, Spark, Hadoop, Big data, Data Science, UNIX and SQL Technologies. Out of which seven years' of Solid experience in Banking Domain. Highly organized and efficient in fast - paced multitasking environments; Strong ability in prioritizing effectively to accomplish objectives with commitment and enthusiasm
  • Extensively worked on Python Programming for Automation and connecting different ecosystems
  • Robust experience in Niche technologies like Apache Spark for Fast Data Processing
  • Best working experience of Big Data Technologies like Hadoop, Hive, YARN, Spark, Datasets, Streaming
  • Strong Experience in Data Science programming and their ecosystems
  • In Depth involvement in Data Movement, Data Transformation and Data Engineering from Start to End
  • Impressive expertise in writing Hadoop Jobs for analyzing data using Hive Query Language
  • Solid experience with various Hive performance tuning techniques like Partitioning and Bucketing.
  • Experience in importing and exporting data between HDFS and RDBMS using SQOOP
  • Possess High Enthusiasm in penetrating into uncharted world of Artificial Intelligence like Deep Learning
  • Exceptional skills in Automating repetitive tasks using Python, UNIX Bash Shell Scripting, AutoSys Job Scheduler
  • Highly proficient in writing Database related scripts like SQL Scripts which include Stored Procedures, Functions, Triggers, Packages, Indexes
  • Solid Experience in Investment Banking, Insurance, Telecom and Government Sector Domains
  • Excellent ability on providing Innovative solutions to business needs.
  • Strong judgment and decision-making abilities

TECHNICAL SKILLS:

Programming languages: Python, PySpark, Shell Scripting, SQL, PL/SQL and UNIX

Big Data: Hadoop, Hive, Sqoop, Apache Spark, Fast Data, Map Reduce

Operating Systems: UNIX, LINUX, Solaris, Mainframes

Data bases: Oracle, DB2, Sybase, Netezza

IDE Tools: Aginitiy for Hadoop, PyCharm, Toad, SQL Developer, SQL *Plus, Sublime Text

Others: AutoSys job Scheduler, ArcGIS, Clarity, Informatica, Business Objects, IBM MQ, Splunk

PROFESSIONAL EXPERIENCE:

Confidential, Plano, TX

Big Data Engineer

Responsibilities:

  • All round working knowledge on Python programming in making it the back bone to connect systems
  • Worked on SQOOP scripts to transfer data from RDBMS to Hadoop Environment
  • Extensively worked on Spark Scripts to transform data in Hadoop and make it utilizable for Machine Learning
  • Strong working knowledge on Spark for fast movement of transformed data into final layer
  • Seamlessly worked on Python to build data pipelines after the data got loaded from Kafka
  • Robust working experience on Spark Streaming for Real Time Data Processing
  • Extensively worked on Hive Queries optimizations and reduced time from 20 minutes to 2 minutes
  • Efficiently utilized the Hive optimization techniques such as Partitioning , Bucketing and Explain Plan
  • Complete knowledge of the Big Data Hadoop ecosystem
  • Proposed High Level standards in implementation and executed across the team
  • Seamlessly able to convert hard-to-grasp technical requirements into outstanding designs
  • Involved from Start to End processes of Converting complex SAS reports to Hadoop scheduled jobs

Technical Environment: Python, Apache Spark, Hadoop Horton Works, SQOOP, Hive, Kafka UDB and SAS

Confidential, Alexandria, VA

Big Data Engineer

Responsibilities:

  • Loaded data from Database into Hive systems
  • Implemented High-speed querying using Spark , Hive and SQL
  • Wrapped these scripts in Python Programming from start to end
  • Efficiently utilised PySpark to create datasets for Data Analytics
  • Focussed on performance tuning of Hive/Spark SQL for faster results
  • Translated complex functional and technical requirements into detailed design.
  • Lead the phase of providing data insights using Hive , Spark , DB, UNIX to Business

Technical Environment: Python, Hadoop, Hive, Apache Spark, Sqoop, Oracle and UNIX

Confidential

Hadoop Developer

Responsibilities:

  • Highly involved in creating Hive tables, loading data, writing hive queries, generating partitions and buckets for optimization.
  • Importing the data from Oracle into the HDFS using Sqoop . Performed full and incremental imports using Sqoop jobs.
  • Responsible to manage data coming from various sources and involved in HDFS
  • Fine-tune the PySpark codes for the optimized utilization of Hadoop resources for the production run.
  • Automating repeated tasks using Python and UNIX Bash Scripting
  • Extensive Experience in writing Hive queries and Fine Tuning them
  • Robust utilization of Apache Spark to transform Bulk data for further Data Munching
  • Written Python scripts for movement of data across different systems
  • In Depth working knowledge on AutoSys job schedulers to automate Python Programs
  • Constructively involved in data modeling of several applications
  • Extensively utilized Explain Plan, Oracle hints and creation of new indexes to improve the performance of SQL statements
  • Flexible in providing workday and after-hours production support for applications.

Technical Environment: Python, Hadoop Horton Works, Sqoop, Hive, Apache Spark, Oracle, SQL and UNIX

Confidential

Senior Data Analyst

Responsibilities:

  • Ensured In depth Data Analysis of all the Business requests raised
  • Automated Data retrieval involving complex systems such as Mainframes, DB2 and UNIX using Bash Shell Scripting
  • Highest Standards were ensured for providing requested data for Bank Operation Users
  • Efficiently fine tuned the query before running the job in Mainframes as it involves cost for each query run
  • Best Analysis provided for the Vendor data feeds like Broadridge, Omgeo to the Bank and their internal data flow within Bank Production Applications
  • Effectively managed the risks to ensure the security and resiliency of the application are in compliance with firm and regulatory requirements
  • Complete ownership of the tasks assigned until timely completion

Technical Environment: Oracle 11g, PL/SQL, SQL, UNIX, Python, AutoSys, DB2, Mainframes, Toad, Service now, Jenkins, SVN, GitHub, IBM MQ, XML

Confidential

Senior Analyst

Responsibilities:

  • Generating SQL Scripts for Database Objects like Sequences, Views and Indexes.
  • Working in Clarity Tool to interact with the Database
  • Efficiently worked on Telecom Data involving UNIX and provided high level reports
  • Working in Unix Environment to interact with the Physical Telephone Switches.
  • Working on Oracle Database to ensure the proper re-connection and Disconnection of Telephone Numbers.
  • Participated in Business Requirements and Functional Requirements meetings identify gaps in requirements and drive discussion around appropriate solutions.

Technical Environment: Oracle 10g/9i, PL/SQL, SQL, PL/SQL Developer, Clarity Tool

Confidential, Lansing

Software Analyst

Responsibilities:

  • Extensively developed new features with analysis, coding and testing in Oracle PL/SQL.
  • Developed PL/SQL functions, procedures, triggers, and packages.
  • Created database Tables, Views, Sequences in Development and Production environment
  • Used Exception Handling extensively for the ease of debugging and displaying the error messages in the application.
  • Imported data through various environments by using SQL * Loader.
  • Created IN and out bound flat files like CSV, TXT and exported to another database.

Technical Environment: Oracle 11g/9i, PL/SQL, SQL, SQL *Loader, SQL Developer, SQL *PLUS, UNIX, Tortoise SVN

We'd love your feedback!