We provide IT Staff Augmentation Services!

Bigdata Lead Technical Consultant / Architect Resume

Richmond, VA

SUMMARY

  • 11+ years of experience in all phases of Analysis, Design, Development, Testing and Support in Middleware, ETL and Big Data technologies (Hadoop and Spark)
  • 3+ years of experience in Big Data/Hadoop and its components like HDFS, MapReduce, Hive, Pig, Sqoop, Hbase, Flume, Oozie and Java
  • 2+ years of experience in Spark, Scala and Kafka
  • Good hands on experience on Spark stack (Spark - SQL, Spark-Streaming) for processing large volume of data.
  • Having good knowledge in Spark-GraphX and Spark-MLlib and knowledge in Machine learning algorithms
  • Familiar with creating the statistical graphs and dashboards using QlikView.
  • Proficient in Analyzing Business processes requirements and translating them into technical requirements. And good exposure to business teams across the globe.
  • Hands on experience on data migration from ETL and Mainframe systems to Hadoop.
  • Currently I’m working as Lead technical consultant for Data migration analysis, design and development Big data Processing Scripts using Hadoop frame work.
  • Having hands on experience in Control M job scheduling.
  • As a Technical lead my responsibilities includes requirement gathering, Solution design, preparing HLDS, support and implementation of deliverables.
  • I also hold responsibility of providing the Project estimates, project planning and Tracking of the deliverables.
  • I was given long term assignments in Blujay Solutions Nordic A/S, Horning, Denmark to work closely with business and product teams to gather the requirements for key customer implementation.
  • Worked with heterogeneous data sources like Oracle, DB2, Flat Files, JSON and XML Files.
  • Experience in various domains like Supply Chain Management and Logistics and Banking.
  • Involved in Migration, Enhancement, Maintenance and Support of projects.
  • Proven ability to learn quickly and apply new technologies and hardworking ability to work in a paced environment.
  • Strong Team working spirit and relationship management skills.
  • I have exposure to project deliveries using in agile methodologies and Project Management and technical discussion with client and daily communication calls for project analysis specs and development aspects.

TECHNICAL SKILLS

Big Data Ecosystems: MapReduce, HDFS, HBase, Zookeeper, Hive, Pig, Sqoop, Oozie, and Flume.

Streaming Technologies: Spark-Streaming, Kafka and Flume.

Scripting Languages: Scala

Programming Languages: Cobol ILE, CL ILE and Java

Databases: DB2, Oracle, NoSQL

IDA’sIntelli: J, Eclipse

Distributed/ETL Tools: Presto, Teradata.

Methodologies: Agile

Operating Systems: Windows, Unix/Linux, OS/400

Job Scheduling tools: Control M

Machine Learning Skills: (MLlib)Feature Extraction, Dimensionality Reduction, Model Evaluation, Clustering.

PROFESSIONAL EXPERIENCE

Confidential, Richmond, VA

Bigdata Lead technical consultant / Architect

Responsibilities:

  • Interacted directly with business for data analysis and implementing the business requirements.
  • Responsible for requirements gathering, Analysis, Design, Development, testing, deployment and post production support for change request/new service request are carried out.
  • Involved in Architectural data work flow discussions
  • Involved in Functional /Technical Specification Document preparation and Review
  • Analyzing the data and giving recommendations to Product Manager for business improvement.
  • Involved in Hadoop cluster estimations
  • Ingesting the raw data onto HDFS cluster from different data sources.
  • Using TDCH wrapper and SQOOP, Import/Export From/To Teradata and Hadoop.
  • Cleansing and checking the quality of the data using hive.
  • Scripting the whole data pipeline using Java/Scala.
  • Working on Spark-SQL and Hive to check the pirated data.
  • Using Bitbucket version control tool for commit the code and Scheduling the Control M jobs and monitoring the jobs execution.
  • Prepare the independent deliverable tasks and assign them to the team
  • Conduct daily standup meetings to know the status of the tasks and help the team to resolve any obstacles in completing tasks and code reviews
  • Unit test plan and test cases review
  • Prepare Application Installation Guide and provide walk thru to implementation teams and production support teams
  • Represent the project change requests in CAB meeting to get the approval for implementation

Environment: Hadoop, Hive, Spark, Sqoop, Hbase, Scala, Teradata, UNIX/LINUX

Confidential, Chelmsford, MA

Senior Hadoop technical consultant / Architect

Responsibilities:

  • Interacted directly with business for data analysis and implementing the business requirements.
  • Responsible for requirements gathering, Analysis, Design, Development, testing, deployment and post production support for change request/new service request are carried out.
  • Created functional and technical specifications based on the requirements.
  • Involved in Architectural data work flow discussions
  • Involved in Functional /Technical Specification Document preparation and Review
  • Analyzing the data and giving recommendations to Product Manager for business improvement.
  • Involved in Hadoop cluster estimations
  • Ingesting the raw data onto HDFS cluster from different data sources.
  • Listening to the events created in server using Kafka.
  • Querying to get the metrics from both structured and unstructured data.
  • Cleansing and checking the quality of the data using hive.
  • Scripting the whole data pipeline using Java/Scala.
  • Working on Spark-SQL and Hive to check the pirated data.
  • Creating graphs/Dashboards from the cleansed metrics data using QlikView
  • Prepare the independent deliverable tasks and assign them to the team
  • Conduct daily standup meetings to know the status of the tasks and help the team to resolve any obstacles in completing tasks and code reviews
  • Unit test plan and test cases review
  • Prepare Application Installation Guide and provide walk thru to implementation teams and production support teams
  • Represent the project change requests in CAB meeting to get the approval for implementation

Environment: Hadoop, Hive, Spark, Sqoop, Hbase, Kafka, Scala, Java, Teradata, UNIX/LINUX, QlikView

Confidential

Lead Hadoop technical consultant

Responsibilities:

  • Gathering and Analysis of the customer business requirements and prepared the KPI’s
  • Involved in the design of application data model.
  • Design and provide the guidance to co developers in the application development
  • Analyzing the business process and client requirements and modifying data models according to the client requirements
  • All financial data has been stored in client’s server at data Centre the sensor data is cleansed and performed analysis to predict the future business.
  • Architecture involves AS/400 DB2 Server for receiving the all shipment bookings data and ETL is being performed by using Sqoop for data migration from DB2 to HDFS, Hive for analysisand QlickView for Data Visualization.
  • Installed and configured Hive and also written Hive-QL scripts.
  • Writing Hive queries for creating external tables, storing and processing the data for further analysis on Visualization tools.
  • Co-ordinating with the business teams for understating requirements weekly.
  • Imparting guidance to the associates on all development areas.

Environment: Hadoop, Apache Pig, Hive, Hbase, Sqoop, DB2, Java, UNIX/LINUX

Confidential

Lead Technical Consultant

Responsibilities:

  • Solution design for the requirements and Prepare High Level design documents
  • Low Level Design Reviews
  • Code Reviews
  • Helping team both technically and domain level understanding of the requirements
  • Test and Test case reviews
  • Coding programs for all major and critical projects to deliver the projects in time
  • SPOC for all design conference/calls related to this application
  • Conducting weekly status review meetings
  • Lead the team in all difficult situations from the front
  • Conducting regular knowledge sharing sessions for the team
  • Preparing the MOM and review of Weekly and Monthly Highlight report which are circulated to Customer
  • Conducting regular Knowledge sharing sessions for the team to make every team member work independently on every task
  • Lead the team in all situations and motivate them towards high productivity
  • Coordinate with client for all issues and concerns
  • Analysis of historical Bridge cases once in 6 months and identify the fixes that would reduce the incidents

Environment:OS/400, COBOL/400, DB2/400 and CL/400, Java

Confidential

Senior Development Lead

Responsibilities:

  • Analysis, Design and Coding of any change requests that come as part of enhancements of the application.
  • Mainly involved in the accounting module changes including Invoices transfer to accounting Analysis, Design and Coding of any change requests that come as part of enhancements of the application.
  • Mainly involved in the accounting module changes including Invoices transfer to accounting systems, account receivables & payables (ARP), Dossier system and General ledger module
  • In ARP module, mainly involved for the programs related to Bank payment format to update the files into online banking system.
  • Change requests have been handled through CMS tool like Turnover to deliver the changes to customer.
  • Analyzing the business process and client requirements and modifying the modules according to the client requirements
  • Programming & Testing the Application according to the requirements
  • Debugging and Testing of application programs.
  • Involved in preparation of test plans and Effort Estimates for the new requirements
  • Analyzing and fixing bugs in the existing system
  • Submitting weekly status reports
  • Perform Unit testing and Preparing of unit test cases
  • Discussion with Application users for their issue-clarifications.
  • Coordinating and monitoring the development activities.
  • Preparing the weekly and monthly project status report for Customer and higher management

Environment: OS/400, COBOL/400, DB2/400 and CL/400

Confidential

Development Lead

Responsibilities:

  • Analysis, Design and Coding of any change requests that come as part of enhancements of the application.
  • Work closely with Product owners to provide the inputs to achieve best possible solution
  • Prepare High level estimates and project plans and delegate projects to the team and help them to handle independently.
  • Prepare DB2 design documents for the table changes and code reviews
  • Analyzing the business process and client requirements and modifying the modules according to the client requirements
  • Programming & Testing the Application according to the requirements
  • Debugging and Testing of application programs.
  • Involved in preparation of test plans and Effort Estimates for the new requirements
  • Analyzing and fixing bugs in the existing system
  • Submitting weekly status reports
  • Perform Unit testing and Preparing of unit test cases
  • Discussion with Application users for their issue-clarifications.
  • Coordinating and monitoring the development activities.
  • Preparing the weekly and monthly project status report for Customer and higher management

Environment: OS/400, COBOL/400, DB2/400 and CL/400

Confidential

Senior Software Engineer

Responsibilities:

  • Analysis, Design and Coding of any change requests that come as part of enhancements of the application.
  • Analyzing the business process and client requirements and modifying the modules according to the client requirements
  • Programming & Testing the Application according to the requirements
  • Debugging and Testing of application programs.
  • Involved in preparation of test plans and Effort Estimates for the new requirements
  • Analyzing and fixing bugs in the existing system
  • Submitting weekly status reports
  • Perform Unit testing and Preparing of unit test cases
  • Discussion with Application users for their issue-clarifications.

Environment: OS/400, COBOL/400, DB2/400 and CL/400

Confidential

Software Engineer

Responsibilities:

  • Analysis, Design and Coding of any change requests that come as part of enhancements of the application.
  • Analyzing the business process and client requirements and modifying the modules according to the client requirements
  • Programming & Testing the Application according to the requirements
  • Debugging and Testing of application programs.
  • Involved in preparation of test plans and Effort Estimates for the new requirements
  • Analyzing and fixing bugs in the existing system
  • Submitting weekly status reports
  • Perform Unit testing and Preparing of unit test cases
  • Discussion with Application users for their issue-clarifications.

Environment: OS/400, COBOL/400, DB2/400 and CL/400

Hire Now