We provide IT Staff Augmentation Services!

Hadoop Developer Resume

3.00/5 (Submit Your Rating)

Irving, TexaS

PROFILE:

  • 14 years of IT industry experience encompassing a wide range of skill set, roles and industry verticals.
  • Extensive experience with analysis, design, development, customizations and implementation of software applications including Mainframe applications.
  • Providing overall management, coordination, and oversight of all facets related to Fidelity Omni Plus and Omni Trade applications.
  • Adept in analyzing information system needs, evaluating end user requirements, custom design solutions, troubleshooting for complex information systems
  • Proven success in Project Management activities with demonstrated abilities in software development.
  • In depth expertise in all aspects of the Agile and SDLC including requirement analysis, Design, development, testing, troubleshooting and debugging
  • Good communication skills, interpersonal skills, self - motivated, quick learner, team player.
  • Strong skills on PL/SQL Windows Scripting, SAS, Hadoop, Python, Scala, Hive, Pig, Flume, HDFS knowledge.
  • Experienced working on AWS environment (cloud services using S3 storage, Datastax enterprise package with Spark and Cassandra NoSQL database).
  • Involved in Cassandra database design, decisions for choosing right table partition, clustering keys for the efficient storage and better performance while reading/lookup to the table.
  • Experience in working on spark streaming API to process the streaming data.
  • Developed workflow in OOZIE to manage and schedule jobs on Hadoop cluster for extracting data from source on daily, weekly, monthly, quarterly and annually basis.
  • Developed data transformations as per business logic using Python and executed the code in Spark by running the Pyspark code upon embedding them in shell scripts.
  • Experience on Hadoop Ecosystems Hive, Pig, Sqoop, Impala, HBase, Flume, Storm, Oozie and Zookeeper. Experience in writing HiveQL queries and Pig Latin scripts. Created UDFs for performing the data transformations as per the requirements.

TECHNICAL SKILLS:

Big Data Technologies: Spark, Hadoop (HDFS, MapReduce), SQOOP, Hive, Pig, Flume, Oozie, Scala, Python, Core-Java, Spark SQL, Storm, Zookeeper.Oozie,Qlik,Oralce,Unix shell scripting.

Development Tools: Interest, Endeavor, File-Aid, File-Master,SPUFI,QMF,SORT, Infoman,Changeman,SARA,ISPF,Comparex,XML Test Harness Tool,Datapower.

Functional Modules: Finance Omni plus, Omni trade Modules, Distribution/Order Management, Warehousing, and Retail Industries.

RDBMS: Oracle 10g, DB2

Languages: PL/SQL, REXX, Python, Scala, Hive, Pig, Java.

O/S: Z/OS, OS/390, Windows 7, Windows Vista, UNIX, Linux

Processes: IBM’s QMS (Quality Management System), OPAL, Jira

PROFESSIONAL EXPERIENCE:

Hadoop Developer

Confidential, Irving, Texas

Responsibilities:

  • Responsible for data ingestion framework & data modeling for ERP, Supply chain & Non-ERP data and process them into finance data lake by using Hadoop Big Data platform.
  • Design and perform Oracle/ETL development
  • Development on Linux systems and UNIX shell scripting as per the business requirements.
  • Work with Big Data technologies, distributed file systems, Hadoop, HDFS, Hive, Hbase, spark, Hive or Pig scripting
  • Work with other database developers to provide database performance-tuning expertise as needed.
  • Providing the customized solution to the Standard to achieve their business goals. data transformation and data transfer between RDBMS & distributed file systems
  • Involve in planning space allocation, creating and managing table spaces Develop and document test plans for new and modified applications covering Unit Testing and System Testing.
  • Import data from multiple source systems (data sources), using SQOOP and analyzed data using Hive and Pig and performed transformations as per the requirements and exporting of the data to down streams.
  • Support code/design analysis, strategy development and project planning.
  • Create Hbase tables, Integrated Hive with Hbase as per the business requirements.
  • Work on HiveQL queries and Pig Latin scripts for the transformations.
  • Work with lead architect and design the data ingestion & data modeling frame work for future business needs.
  • Coordinating with the Business users and being the point of contact between the business user and the team, gather requirements, attend daily touch-point calls, update the business users on the latest developments and deliver the tasks assigned to me and the team in a timely fashion and accurately.
  • Implement the tool designed by the DevOps team to migrate the code and scripts to non- production environments via JIRA Tickets.
  • Write complex SQL queries and designed ETL flows to load data into Impala and leveraged OOZIE to run ETL flows.

Environment: Hadoop HDFS, MAP Reduce, Pig, Hive, Sqoop, Oozie, Hbase, Spark, SparkSQL, Python, Kafka, Sprak Streaming, Scala, Eclipse, Web Services, Windows scripting, Unix shell scripting,Java,Qlik.

Data Engineer

Confidential, Wallingford, CT

Responsibilities:

  • Analyze the current system and new PRIME system in WGS environment to build data mapping and creating cross walk programs from ACES to WGS system and vice-versa.
  • Modify all existing applications to enable smooth migration to PRIME environment.
  • Create advanced procedures like text analytics, processing using in memory computing capabilities like Spark written in Scala.
  • Work on with data streaming process with Kafka, Apache Spark, Hive, Pig, etc.
  • Write a Python module to connect and view the status of an Apache Cassandra instance.
  • Perform in-memory data processing and real time streaming analytics using Apache Spark with Scala and Java.
  • Assist and troubleshoot current Mainframe environment (ACES) and new Mainframe environment (PRIME) by utilizing his technical knowledge.
  • Design, code, test, debug, and record those programs proficient to work at top technical level of every level of applications programming activities.
  • Evaluate, design, code and test improvement as required to complex modules.
  • Design and develop code applications to technical and functional programming standards.
  • Provide prime assistance toward application releases installation into production.
  • Plan and implement every needed process step as defined in methodologies.
  • Develop operational documents for Application.
  • Present application and technical support as necessary at right time.
  • Provide estimate of effort and resource allocation to client manager to enter in plan view tool.
  • Coordinate analysis and resolution of defects raised by QA team for project and track closure of the same in JIRA tool.
  • Worked on importing of data from multiple source systems (data sources), using SQOOP and analyzed data using Hive and Pig and performed transformations as per the requirements and exporting of the data to down streams.

Environment: Hadoop, Hive, Pig, SQOOP, Kafka, Spark, OOZIE, Python, SCALA, UNIX, Shell scripting, Oracle PL/SQL, RDBMS, AWS, Oracle Golden Gate, Kyvos, Tableau/Qlik.

Technical Lead

Confidential, Portland, OR

Responsibilities:

  • Revenue sharing project: Provide a system that allocates revenue sharing received for NAV-based plans back to the participants as well as provide a fully-auditable and reportable process to accurately track distribution of participant-level revenue-sharing amounts.
  • Discussion with Business and the external vendors for the requirement gathering and conducting the scrum calls with team.
  • Coordinating with offshore team for the clarification and explaining the requirements and creating the user cases.
  • Design and develop code applications to technical and functional programming standards.
  • Providing the customized solution to the Standard to achieve their business goals.
  • Utilize in-depth knowledge of functional and Technical experience in Mainframe, PL/SQL, Windows Scripting and other Omni channels leading-edge products and technology in conjunction with industry and business skills to deliver solutions to customer.
  • Develop and document test plans for new and modified applications covering Unit Testing and Facilitate Onshore-Offshore calls among Customer and Offshore resources to understand the requirement and to approve the approaches designed.
  • Supported code/design analysis, strategy development and project planning.
  • Worked on HiveQL queries and Pig Latin scripts for the transformations.
  • Managing and reviewing Hadoop Log files and optimizing the code.
  • Writing Spark-API for comparing the both Cassandra and DB2 Table data.

Environment: Linux, Windows, Hadoop, Hive, Pig, SQOOP, Kafka, Spark, OOZIE, Python, SCALA, UNIX, Shell scripting, RDBMS, Oracle PL/SQL, Tableau/Qlik

Technical Lead

Confidential, Bentonville, AR

Responsibilities:

  • Design, code, test, debug, and record those programs proficient to work at top technical level of every level of applications programming activities.
  • Evaluate, design, code and test improvement as required to complex modules.
  • Design and develop code applications to technical and functional programming standards.
  • Provide prime assistance toward application releases installation into production under given direction.
  • Utilize in-depth knowledge of functional and Technical experience in Mainframe and other leading-edge products and technology in conjunction with industry and business skills to deliver solutions to customer.
  • Present application and technical support as necessary at right time.
  • Develop and document test plans for new and modified applications covering Unit Testing and System Testing.
  • Developed multiple spark batch jobs in Scala using Spark SQL and performed transformations using many APIs and update master data in Cassandra database as per the business requirement.
  • Written spark-scala scripts, by creating multiple udf’s, spark context, Cassandra sql context, multiple API’s, methods which support data frames, RDD’s, data frame Joins, Cassandra table joins and finally write/save the data frames/RDD’s to Cassandra database.
  • Involved in processing the huge number of inbound feed files using Spark batch Jobs and update the master data in Cassandra tables.

Environment: AWS, Spark, SparkSQL, Scala, Spark Standalone, Amazon Workspace, Cassandra, IntellJ IDE, Git, Putty to connect to Cassandra and Spark instances, CICS, COBOL, DB2, JCL, VSAM, CICS Web Services, Endevor, Datacom, File Master, SPUFI,jORACLE, PL/SQL,AS/400,,SAS.

Team Lead

Confidential

Responsibilities:

  • Responsible for leading a project team in delivering solution to our customer in the retail sector.
  • Deliver new and complex high-quality solutions to clients in response to varying business requirements
  • Responsible for managing scope, planning, tracking, change control, aspects of the project.
  • Responsible for effective communication between the project team and the customer. Provide day to day direction to the project team and regular project status to the customer.
  • Translate customer requirements into formal requirements and design documents, establish specific solutions, and leading the efforts including programming and testing that culminate in client acceptance of the results.
  • Utilize in-depth knowledge of functional and Technical experience in Mainframe and other leading-edge products and technology in conjunction with industry and business skills to deliver solutions to customer.
  • Establish Quality Procedure for the team and continuously monitor and audit to ensure team meets quality goals.
  • Develop and document test plans for new and modified applications covering Unit Testing and System Testing.

Environment: CICS, COBOL, DB2, JCL, VSAM, CICS Web Services, Endevor, Datacom, File Master, SPUFI, ORACLE, PL/SQL

Confidential

Technical Analyst

Responsibilities:

  • Work with Business Analyst in translating business requirements into Functional Requirements Document and to Detailed Design Documents
  • Lead analysis sessions gather requirements and write specification and functional design documents for enhancements and customization; Analyze product impact
  • Co-ordinating offshore team and updating status of work to client
  • Responsible for interacting directly with the Client manager and conducting same time meetings with client-side representatives.
  • Ensure that development is performed as per requirements
  • Involved in helping other team members in providing Problems and Performance improvements
  • Communicate activities/progress to project managers, business development, business analysts and clients

Environment: CICS, COBOL, DB2,JCL,VSAM, CICS Web Services, Endevour, File Master, CA Interest, XML, XSLT,WSDL,SOAP, Data power

Asst.Systems Engineer

Confidential, Long Island, NY

Responsibilities:

  • Design, develop, test and implement, modify and maintain software systems for various business application projects using COBOL, Ideal, CICS, Datacom, and DB2 in various environments like IBM Mainframes.
  • Build applications using COBOL, Ideal, Easytrieve, Datacom, DB2, CICS, VSAM and JCL.
  • Modify existing software to adapt to new requirements
  • Monitor critical jobs of application.
  • Perform regular administrative functions dealing with problem tracking/management and peer review processes.
  • Produce User documentation for new applications outlining functionality, general decision logic and data flows.
  • Provide technical and investigative support for functional users on interfaces, defects, change requests, debugging issues, testing
  • Conduct reviews of all deliverables and maintain records of reviews on the defect-tracking database.
  • Prepare High and low-level designs based on requirements.
  • Provide Production support and maintenance.
  • Provide 24X7 on call application support.
  • Create First Choice Change tickets for Code Promotion, Database Table changes.etc
  • Replicate the Test databases with Production databases.
  • Attend project meetings and quality review meetings.

Environment: Win CICS, COBOL, DB2, JCL, VSAM, Change man, Infoman, File Aid, Xpeditor, CA Scheduling

We'd love your feedback!