We provide IT Staff Augmentation Services!

Big Data Developer Resume

Atlanta, GA

SUMMARY

  • 8 years of professional IT experience working in development, testing and support of Web based Application including 2 years of experience in Big Data Ecosystem.
  • Extensive knowledge on e - commerce systems implementation.
  • Cloudera certified Hadoop Developer.
  • Familiarity wif Agile development including daily scrum and weekly iteration reviews and planning.
  • Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase database, Spark and Sqoop.
  • Worked in converting Hive/SQL queries into Spark transformations using Spark RDDs and Data Frames.
  • Hands on experience in Hadoop components like HDFS, MapReduce, Job Tracker, Name Node, Data Node and Task Tracker.
  • Extending Hive and Pig core functionality by writing custom UDFs.
  • Good experience in analyzing data using Hive Query Language, Pig Latin and custom MapReduce programs in Java along wif using User Defined Functions.
  • Involved in loading data from various sources to HDFS, using Sqoop, Flume and Kafka.
  • Developed Kafka producer and consumers.
  • Worked on creating, modifying topics wif varying configurations involving replication factors and partitions.
  • Developed Spark and Hadoop MapReduce jobs.
  • Cluster coordination services through Zookeeper.
  • Worked on teh core and Spark SQL modules of Spark.
  • Extensively worked on Text, ORC, Avro and Parquet file formats and compression techniques like Snappy, Gzip and Zlib.
  • Worked on Sequence files, RC files, and Map side joins, bucketing, partitioning for Hive performance enhancement and storage improvement and utilizing HiveSerDes like REGEX, JSON and AVRO.
  • Used OOZIE Operational Services for batch processing and scheduling workflows dynamically.
  • Writing unit test cases for teh developed component and conducting code reviews and correcting teh code according to teh review comments.

TECHNICAL SKILLS

  • Core Java
  • Java 1.6
  • JSP
  • Servlets
  • Spring MVC
  • Hibernate
  • Web Services
  • XML
  • JNDI
  • JDBC
  • Spring
  • Oracle WebLogic Application Server 10.3
  • Oracle 9i
  • UML
  • Eclipse
  • HTML
  • CSS
  • AJAX
  • JavaScript
  • Ant
  • JUnit.

PROFESSIONAL EXPERIENCE:

Confidential, Atlanta, GA

Big Data Developer

Responsibilities:

  • Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Python and Scala.
  • Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase database, Spark and Sqoop.
  • Developed multiple POCs using spark and deployed on teh Yarn cluster, compared teh performance of Spark, wif Hive and SQL/Teradata.
  • Have utilized Python libraries such as Numpy and Pandas for processing tabular format data.
  • Involved in loading data from various sources to HDFS.
  • Worked on installing cluster, commissioning & decommissioning of datanode, namenode recovery, capacity planning, and slots configuration.
  • Developed Kafka producer and consumers.
  • Created DW data model in HBase and RDBMS for data aggregation and active analytic.
  • Load and transform large sets of structured, semi structured and unstructured data.
  • Experience in managing and reviewing Hadoop log files.
  • Created scripts for system administration and AWS using languages such as BASH and Python.
  • Exported teh analyzed data to teh relational databases using Sqoop for visualization and to generate reports for teh BI team.
  • Installed Oozie workflow engine to run multiple Hive and pig jobs.
  • Involved in loading data from various sources to HDFS, using Sqoop, Flume and Kafka.
  • Worked on creating, modifying topics wif varying configurations involving replication factors and partitions.
  • Developed Spark and Hadoop MapReduce jobs.
  • Cluster coordination services through Zookeeper.
  • Worked on teh core and Spark SQL modules of Spark.
  • Extensively worked on Text, ORC, Avro and Parquet file formats and compression techniques like Snappy, Gzip and Zlib.
  • Implemented Hortonworks NiFi (HDP 2.4) and recommended solution to inject data from multiple data sources to HDFS and Hive using NiFi.
  • Developed various data loading strategies and performed various transformations for analyzing teh datasets by using Hortonworks Distribution for Hadoop ecosystem.
  • Ingested data from RDBMS and performed data transformations, and then export teh transformed data to Cassandra as per teh business requirement and used Cassandra through Java services.
  • Worked in NoSQL Column-Oriented Databases like Cassandra and its Integration wif Hadoop cluster.
  • Created Partitions, Buckets based on State to further process using Bucket based Hive joins.
  • Worked on Sequence files, RC files, and Map side joins, bucketing, partitioning for Hive performance enhancement and storage improvement and utilizing HiveSerDes like REGEX, JSON and AVRO.
  • Used OOZIE Operational Services for batch processing and scheduling workflows dynamically.

Environment: Spark, Kafka, Sqoop, Hadoop, Mapreduce, Cassandra, Hive, HBase, Pig, Oozie, Flume, HDP, Java, Scala, Python, and Eclipse.

Confidential, Atlanta, GA

Big Data Developer

Responsibilities:

  • Involved in HDFS maintenance and loading of structured and unstructured data. Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase database and Sqoop.
  • Responsible for building scalable distributed data solutions using Hadoop.
  • AWS Cloud management and Deployment.
  • Implemented nine nodes Horton works Hadoop cluster on Red hat LINUX.
  • Involved in loading data from LINUX file system to HDFS.
  • Worked on installing cluster, commissioning & decommissioning of datanode, namenode recovery, capacity planning, and slots configuration.
  • Created HBase tables to store variable data formats of PII data coming from different portfolios.
  • Implemented a script to transmit sysprin information from Oracle to Hbase using Sqoop.
  • Implemented best income logic using Pig scripts and UDFs.
  • Implemented test scripts to support test driven development and continuous integration.
  • Worked on tuning teh performance Pig queries.
  • Worked wif application teams to install operating system, Hadoop updates, patches, version upgrades as required.
  • Responsible to manage data coming from different sources.
  • Involved in loading data from UNIX file system to HDFS.
  • Load and transform large sets of structured, semi structured and unstructured data
  • Cluster coordination services through Zookeeper.
  • Experience in managing and reviewing Hadoop log files.
  • Job management using Fair scheduler.

Environment: Pig, Hive, Hadoop, Sqoop, Mapreduce, Oozie, Flume, Java.

Confidential, Atlanta, GA

Java/Hadoop Developer

Responsibilities:

  • me Designed and developed UI using Struts view tags (HTML, Bean, Logic and Nested), JSP, HTML, CSS and Struts Tiles.
  • Configured Struts, Spring, hibernate for teh development environment.
  • Designed and Developed parsers classes using SAX Parsers.
  • Involved in Developing SOAP messages as part of Web Services testing.
  • Developed XSD schemas.
  • Developed XML beans using XSD schemas.
  • Used Web Logic Workshop for development Environment.
  • Configured Data Source in Web Logic Server.
  • Used Subversion as Source code control.
  • Developed Ant scripts for generating teh XML Beans.
  • Generated DAO, POJO classes and Hibernate mapping files using Hibernate tools.
  • Modified DAO classes and Hibernate mapping files as per teh application standard.
  • Used log4j, JUnit for logging and testing teh application.

Environment: Core Java, Java 1.6, JSP, Servlets, Spring MVC, Hibernate, Web Services, XML, JNDI, JDBC, Spring, Oracle WebLogic Application Server 10.3, Oracle 9i, UML, Eclipse, HTML, CSS, AJAX, JavaScript, Ant, JUnit.

Confidential, San Antonio, TX

Java Developer

Responsibilities:

  • Extensively used Hibernate framework to persist teh operations data to teh database.
  • Wrote complex SQL and HQL queries to retrieve data from teh Oracle database.
  • Used Web Services for communication between teh different internal applications.
  • Used Apache Axis as teh Web Services framework for creating and deploying Web Services clients using SOAP and WSDL.
  • Scaled teh component interactions and generalized teh interfaces using RESTFUL.
  • Used SOAP for teh communication between teh different internal applications.
  • Involved in producing necessary WSDL to deploy as a Web Service.
  • Developed teh XML schema and Web Services for teh data maintenance and structures.
  • Developed classes to interface wif underlying web services layer.
  • Designed and developed JSP pages using Spring MVC framework.
  • Used Spring Security for Authentication and authorization extensively.
  • Used Spring Core for dependency injection/Inversion of control (IOC).
  • Developed teh Controller Servlet to handle teh request and responses.
  • Designed rich user interface screens using AJAX, Java Script, CSS, and HTML.
  • Involved in teh design of teh Referential Data Service module to interface wif teh various internal databases using JDBC.
  • Responsible for designing technical specifications by reviewing Business Requirements wif Business Analysts
  • Used Commons and Log4J logging framework. Log messages wif various levels are written in all teh Java code.
  • Writing unit test cases for teh developed component and conducting code reviews and correcting teh code according to teh review comments.

Environment: Core Java, Java 1.6, JSP, Servlets, Spring MVC, Hibernate, Web Services, XML, JNDI, JDBC, Spring, Oracle WebLogic Application Server 10.3, Oracle 9i, UML, Eclipse, HTML, CSS, AJAX, JavaScript, Ant, JUnit.

Hire Now