Sr.hadoop Developer Resume
Phoenix, ArizonA
SUMMARY:
- Over 8 years of experience in software industry doing analysis, design, development, testing, implementation and support of n - tier applications using industry standard methodologies.
- Worked on Hadoop ecosystem(MR1,YARN,HDFS,PIG,Hive,Sqoop,Oozie,HBase,Flume,Zookeeper).
- Successfully delivered couple of initiatives (Implementation & development) on Big Data Analytics and large data processing using Hadoop ecosystem.
- Developed applications using core java, collection framework, JSP, JDBC, Web Sphere, XML, Rest and Soap web services.
- Expertise in implementing complex Business rules by creating robust mappings, mapplets, and reusable transformations using Informatica Power Center and Power Mart.
- Extensive Knowledge in architecture design of Extract, Transform, Load environment using Informatica Power Mart and Power Center.
- Strong knowledge and understanding of data modeling concepts and Cognos Reportnet,.
- Expertise in writing complex and efficient SQL, PL/SQL scripts, Stored Procedures and Functions.
- Used Informatica Workflow Manager to create and schedule workflows and Worklets.
- Proficient in development methodologies such as Agile, Scrum and Waterfall.
- Experience working on high availability and high traffic applications.
- Worked with customers, end users to formulate and document business requirements.
- Proven ability to learn quickly and apply new technologies. Strong problem solving & technical skills coupled with clear decision making.Highly motivated resourceful team player takes a positive approach to problem resolution.
- Worked extensively on Business Requirements Analysis, Functional and Non-Functional requirements analysis, Risk Analysis and UAT.
- Good communication, inter-personal skills, team player and contributor who delivers on schedule under tight schedules.
TECHNICAL SKILLS:
Hadoop Distribution: Apache, Cloudera CDH, HortonWorks HDP
Big Data Technologies: ApacheHadoop(MRv1,MRv2),Hive,Pig,Sqoop,HBase,Flume,Zookeeper, Oozie, AMBARI, Spark, TEZ, KAFKA, Storm,R, Elasticsearch,Solr
Cloud Platforms: Amazon Web Services (EC2), Google cloud platform
Operating Systems: Windows, Linux & Unix
Languages: C,C++, Java, PL/SQL, Unix Shell
Web Technologies: HTML, JSP, JSF, CSS, JavaScript
IDEs: Eclipse, JBOSS, IBM Web Sphere
Reporting Tools: SAP Business Objects, Microstategy, Tableau
WebServers /App Servers: Apache Tomcat 6.0/7.0, IBM WebSphere 6.0/7.0, JBoss 4.3
ERP: SAP R/3 FICO
Database: Oracle 8i/9i/10g/11g, MySQL, SQL Server 2008, MongoDB(NoSQL), LDAP
ETL Tools: Informatica
Defects Triage: HP ALM,Quality Center, BugZilla, JIRA,Microsoft TFS
PROFESSIONAL EXPERIENCE:
Confidential, Phoenix,Arizona
Sr.Hadoop Developer
Responsibilities:
- Responsible for data movement from client library and relational database to HDFS using Sqoop.
- Handled importing data from various data sources, performed transformations using Hive, Map Reduce, and loaded data into HDFS.
- Involved writing Pig Latin for data Processing as per use cases.
- Optimizing the PIG Latin scripts to improve the processing times.
- Involved in Import and Export of data using Sqoop.
- Developed Map Reduce programs in Java for parsing the raw data and populating staging tables.
- Build Hive UDF's for business requirements which enabled users to use these UDF's in Hive Querying.
- Created external tables using hive to access data from reporting tools using hive ODBC connectors.
- Involved in design and development of Tableau reports and dashboards.
- Involved in Agile sprint meeting and scrum meetings. Provided updates in daily SCRUM and Self planning on start of sprint and provided the planned task.
- In sync up with team in order to pick priority task and update necessary documentation
- Worked on design and architecture of the project.
- Worked on Performance tuning on Hive SQLs and pig scripts
- Documenting the Technical details regarding the issues and help to reduce the timeframes in development when same issues occur.
Environment: Hadoop, HDFS, Hive, MapReduce, Sqoop, Oracle Server, LINUX, Cloudera Manager, Ambari, HDP 2.0/2.1, CDH 5.2.1, Hbase, Pig,Oozie,Tableau.
Confidential, Clayton,MO
Hadoop Developer
Responsibilities:
- Analyze EBBS core banking system datasets for CDC transformations.
- Load and transform large sets of structured, semi structured and unstructured data.
- Extracted files from Oracle, and SQL Server through Sqoop and placed in HDFS and processed.Involved in defining job flows, managing and reviewing log files.
- Installing, configuring, and usingHadoopecosystem components likeHadoopMap Reduce, HDFS, Hive, Pig, Sqoop, HBase, Flume and Spark.
- Involved in importing and exporting the data from RDBMS to HDFS and vice versa using sqoop.
- Installed and configured Hive, Pig, Sqoop, Flume and Oozie on theHadoopcluster.
- Networking communication skills and protocols such as TCP/IP, Telnet, SSH.
- Written Hive queries for data analysis to meet the Business requirements.
- Load and transform large sets of structured, semi structured and unstructured data.
- Got good experience with NOSQL database.
- Good understanding/knowledge ofHadoopArchitecture and various components such as HDFS, JobTracker, Task Tracker, NameNode, DataNode, Secondary Namenode.
- Involved in creating Hive tables loading with data and writing hive queries which will run internally in map reduce way.
- Installed and configured Pig for ETL jobs. Written Pig scripts with regular expression for data cleaning.
- Analyzed large amounts of data sets to determine optimal way to aggregate and report on it..
Environment: Hadoop,Cloudera CDH,HDFS, Pig, Hive, MapReduce, Sqoop, Linux,Unix,Flume, Oozie.
Confidential, Grand Rapids, MI
Hadoop Developer
Responsibilities:
- Delivered end to end project. Form requirement gathering to development and testing.
- Ingested data from different sources intoHadoop
- Created external tables with proper partitions for efficiency and loaded the structured data in HDFS resulted from MR jobs.
- Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
- Provided quick response to ad hoc internal and external client requests for data and experienced in creating ad hoc reports.
- Load and transform large sets of structured, semi structured and unstructured data usingHadoop/Big Data concepts.
- Worked hands on with ETL process.
- Outstanding Data analysis skills including Data mapping from source to target database schemas, Data Cleansing and processing, writing data extract scripts/programming of data conversion and researching complex data problems
- Responsible for creating Hive tables, loading data and writing hive queries.
- Extracted the data from sql server into HDFS using the Sqoop.
- Installed Ozzie workflow engine to run multiple MapReduce, Hive and Pig jobs which run independently with time and data availability.
- Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
- Continuous monitoring and managing theHadoopcluster through Cloudera Manager.
- Developed Hive queries to process the data and generate the data cubes for visualizing.
- Created scripts for performing data-analysis with PIG, HIVE in order to do advanced analytics on top of the datasets and compared against each vendor.
- Exported the patterns analyzed back into Teradata using Sqoop
- Evaluated ETL tools like Informatica for data transformation and performance considerations.
Environment: Hadoop, Cloudera CDH,HDFS, Pig, Hive, MapReduce, Sqoop, Linux, Unix, Flume, Oozie,Informatica Power Center
Confidential, Worcester, MA
ETL Developer
Responsibilities:
- Used ETL methodology for complete processing of Data Extraction, transformations and loading steps, in a corporate-wide-ETL Solution using Informatica.
- Used several transformations like source qualifier, expression, lookup, aggregator, Joiner, sequence generator, router and update strategy transformations for complex mappings.
- Used Debugger in troubleshooting the existing mappings.
- Developed stored procedures and functions in PL/SQL.
- Developed and tuned various SQLs to improve the performance both on SQL Server and Oracle.
- Tuned Informatica mappings for better performance using different techniques to provide maximum efficiency and complete performance.
- Developed sessions using Server Manager for loading the data into target database.
- Monitored Workflows and Sessions using Workflow Monitor.
- Designed and developed catalogs that reflected business view of the data to enable other developers to design cubes and reports.
- Modified power cubes and power play reports as per the requirement.
- Monitored workflows and resolved the issues if there is any causing the delays in processing the data.
- Precise Documentation was done for all mappings, workflows and reports.
Environment: Windows NT, Oracle9i, DB2, PL/SQL, Informatica PowerCenter, PowerPlay6.5, Transformer 6.5, SQL Server
Confidential
Cognos Developer
Responsibilities:
- Developing the Power Play and Impromptu Reports.
- Analyzing user requirements, creating catalogs, organizing folders, setting up user classes and security, integrating impromptu with other Cognos products
- Designing the business models and Creating Power Cubes develop a partitioning strategy, build macros using the Cognos script editor.
- Used Cognos Impromptu tool to make a catalog file, tables, joins, folders and profiles and developed the reports
- Involved in analysis of report requirements
- Design and development of technical specifications and mapping documents from the functional requirements.
- Created the reports in Cognos ReportNet using Report Studio and Query Studio with functionalities like CrossTab, Master Detail and Formulae etc.
- Involved in creating charts using Report Studio• Design of test cases and data validation against source database.
Environment: CognosImpromptu 6.0, Cognos PowerPlay,CognosPowerPlay Transformer, SQL Server 2000, MS-Excel, and Windows 2000.