Sr. Java/big Data Developer Resume
3.00/5 (Submit Your Rating)
Nyc, NY
SUMMARY
- Over7+years of experience in Information Technology which includes experience in Big data,HADOOP Ecosystem,Core Java/J2EE and strong in Design, Software processes, Requirement gathering, Analysis and development of software applications
- Excellent Hands on Experience in developing HadoopArchitecturein Windows and Linux platforms.
- Experience in building big data solutions using Lambda Architecture using Cloudera distribution ofHadoop,MapReduce,Cascading,HIVE,PIGandSqoop.
- Strong development experience in Java/JDK 7, JEE6, Maven, Jenkins, Jersey, Servlets, JSP, Struts, Spring, Hibernate, JDBC, Java Beans, JMS, JNDI, XML, XML Schema, Web Services, SOAP, JUnit, ANT, Log4j.
- Experienced inJ2EE Design Patternssuch as MVC, Business Delegate, Service Locator, Singleton, Transfer Object, Singleton, Session Façade, and Data Access Object.
- Worked onHadoop, Hive, JAVA, python, Scala Strutsweb framework.
- Excellent working experience onBig Data Integrationand Analytics basedon Hadoop,SOLR,Spark,Kafka,Stormand web Methods technologies.
- Developed Python code to gather teh data from HBase and designs teh solution to implement using Pyspark
- Experienced in designing and developing applications inSparkusingScalato compare teh performance ofSparkwifHive and SQL/Oracle.
- Worked on Google Cloud Platform(GCP) Services like Vision API, Instances
- Hands on experience working on NoSQLdatabases includingHbase,MongoDB,Cassandraand its integration wifHadoopcluster.
- Strong Knowledge and experience on implementingBig DatainAmazon Elastic MapReduce (Amazon EMR)for processing, managingHadoopframework dynamically scalableAmazon EC2instances.
- Hands on experience in writingAd - hoc Queriesfor moving data fromHDFStoHIVEand analyzing teh data usingHIVEQL.
- Good understanding on Cloud Based technologies such as GCP, AWS.
- Hands on Experience on Snowflake and GCP.
- Good noledge in RDBM Sconcepts(Oracle 11g, MS SQL Server 2000) and strong SQL, PL/SQLquery writing skills (by usingTOAD & SQL Developertools), Stored Procedures and Triggers.
- Expertise in Amazon Web Services including Elastic Cloud Compute (EC2) and Dynamo DB.
- Expertise in Automating deployment of largeCassandra Clusters on EC2 using EC2 APIs
- Experienced in development and utilization of ApacheSOLRwif Data Computations and Transformation for use by Down Stream Online Applications.
- Excellent noledge of database such as Oracle 8i/9i/10g/11g, 12c,Microsoft SQL Server,DB2,Netezza.
- Good understanding and experience wif Software Development methodologies like Agile and Waterfall.
- Experienced in importing and exporting data usingSqoopfromHDFS (Hive & HBase)to Relational Database Systems(Oracle &Teradata)and vice-versa.
- Experienced in developing and designingWeb Services (SOAP and Restful Web services).
- Expertise in various Java/J2EE technologies like JSP,Servlets,Hibernate,Struts,spring.
PROFESSIONAL EXPERIENCE
Confidential, NYC, NY
Sr. Java/Big Data Developer
Responsibilities:
- DevelopedSparkcodeusingScalaandSpark-SQL/Streamingfor faster testing and processing of data.
- UsedSparkAPIoverClouderaHadoopYARNto perform analytics on data in Hive.
- As aBig DataDeveloper implemented solutions for ingesting data from various sources and processing teh Data-at-Rest utilizing Big Data technologies such asHadoop, MapReduce Frameworks, MongoDB.
- Developed a job server(REST API, spring boot, ORACLE DB)and job shell for job submission, job profile storage, job data (HDFS) query/monitoring.
- Developed PySpark and SparkSQL code to process teh data in Apache Spark on Amazon EMR to perform teh necessary transformations based on teh STMs developed
- Created CustomUDF’sin JAVA to overcome HIVE limitations on Cloudera CDH5.
- Explored wif theSparkimproving teh performance and optimization of teh existing algorithms inHadoopusingSparkContext,Spark -SQL, Data Frame,PairRDD's,SparkYARN.
- Deployed application toAWSand monitored teh load balancing of differentEC2instances
- Handled importing of data from various data sources, performed transformations usingHive, MapReduce, loaded data intoHDFSand Extracted teh data fromSQL into HDFS using Sqoop.
- Deployed application toAWSand monitored teh load balancing of differentEC2 instances
- InstalledHadoop, Map Reduce, and HDFSand developed multipleMapReducejobs inPIGandHivefordata cleaning and pre-processing.
- Developed a POC for project migration from on premHadoopMapRsystem to GCP/Snowflake
- Worked on implementing Spark Framework a Java based Web Frame work.
- Worked onBig Data Integration&Analytics based onHadoop, SOLR, Spark, Kafka, Storm and web Methods.
- Extensively worked onPythonand build teh custom ingest framework and w orked onRest API using python.
- DevelopedKafkaproducer and consumers,SparkandHadoopMapReducejobs.
- Imported teh data from different sources likeHDFS/HbaseintoSparkRDD.
- Configured deployed and maintained multi-node Dev and TestKafkaClusters.
- Strongly recommended to bring inElastic Searchand was responsible for installing, configuring and administration.
- Created ElasticMap Reduce (EMR) clusters and Configured teh Data pipeline wif EMR clustersfor scheduling teh task runner and provisioning ofEc2 Instanceson both Windows and Linux.
- Worked onAWS Relational Database Services, AWS Security Groups and their rule andimplementedReporting, Notification services using AWS API.
- Analyzed teh SQL scripts and designed teh solution to implement using Pyspark.
- ImplementedAWS EC2, Key Pairs, Security Groups, Auto Scaling, ELB, SQS, and SNS using AWS API and exposed as teh Restful Web services.
- Involved in convertingMapReduceprograms intoSparktransformationsusingSpark RDD's on Scala.
- DevelopedSparkscriptsby usingScalaShellcommands as per teh requirement.
- Implemented usingSCALA and SQLfor faster testing and processing of data. Real time streaming teh data using wifKAFKA.
- Developed and designed automation framework usingPython and Shell scripting.
- Involved in writingJava APIforAmazon Lambdato manage some of theAWS services.
- Load teh data intoSparkRDDand do in memory data Computation to generate teh Output response.
- DevelopedHive Scripts, Pig scripts, UNIX Shell scripts,programming for allETLloading processes and converting teh files into parquet in theHadoop File System.
- Developed and writtenApachePIGscriptsandHIVEscriptsto process theHDFSdata.
- UsedHiveto find correlations between customer's browser logs in different sites and analyzed them to build risk profile for such sites.
- UtilizedAgile Scrum Methodologyto help manage and organize a team of 4 developers wif regular code review sessions.
Confidential, Nashville, TN
Java/Hadoop Developer
Responsibilities:
- Worked on analyzingHadoop clusterusing different big data analytic tools includingKafka, Pig, HiveandMapReduce.
- Proactively monitored systems and services, architecture design and implementation ofHadoopdeployment, configuration management, backup, and disaster recovery systems and procedures
- ConfiguredSparkstreaming to receive real time data from theKafkaand store teh stream data toHDFSusingScale.
- Installed and configuredHadoop, MapReduce, HDFS (Hadoop Distributed File System),developed multipleMapReducejobs injavafor data cleaning and processing.
- Designed and configured Flume servers to collect data from teh network proxy servers and store toHDFS and HBASE.
- Worked on implementingSparkusingScalaandSparkSQLfor faster analyzing and processing ofdata.
- UtilizedJava and MySQLfrom day to day to debug and fix issues wif client processes
- UsedJAVA, J2EEapplication development skills wif Object Oriented Analysis and extensively involved throughoutSoftware Development Life Cycle (SDLC)
- ImplementedAWS EC2, Key Pairs, Security Groups, AutoScaling, ELB, SQS, and SNS using AWS APIand exposed as teh Restful Web services.
- Monitor Azkaban jobs in on-prem (Hortonworks distribution) and GCP (Google Cloud Platform).
- Involved in launching and Setup ofHADOOP/ HBASECluster which includes configuring different components ofHADOOP and HBASE Cluster.
- Hands-on experience ofWeb logic Application Server, Web Sphere Application Server, Web Sphere Portal Server, and J2EE applicationdeployment technology
- Handled in Importing and exporting data intoHDFSandHiveusingSQOOPandKafka
- Involved in creatingHive tables, loading teh data and writinghivequeries, which will run internally in map reduce.
- AppliedMapReduceframework jobs in java for data processing by installing and configuringHadoop, HDFS.
- Involved in developingPigScriptsfor change data capture and delta record processing between newly arrived data and already existing data inHDFS.
- Developed spark applications in python(PySpark) on distributed environment to load huge number of CSV files wif different schema in to Hive ORC tables.
- Worked on reading and writing multiple data formats like JSON,ORC,Parquet on HDFS using PySpark.
- Involved inHDFSmaintenance andWEBUIit throughHadoop-Java API.
- Implemented Reporting, Notification services usingAWS API and used AWS (Amazon Web services)compute servers extensively.
- WrittenHivejobs to parse teh logs and structure them in tabular format to facilitate effective querying on teh log data.
- Worked on Designing and DevelopingETLWorkflowsusingJavafor processing data inHDFS/HbaseusingOozie.
- Wrote complexHivequeriesandUDFs.
- Create Snapshots ofEBSVolumes. MonitorAWS EC2Instances usingCloud Watchand worked onAWSSecurity Groups and their rules
- Involved in developingShellscriptsto easy execution of all other scripts(Pig, Hive, and MapReduce)and move teh data files wifin and outside of HDFS.
- Involved in convertingHive/SQLqueries into Spark transformations using SparkRDDs, PythonandScala.
- Worked wifNoSQLdatabases likeHbasein creating tables to load large sets of semi structureddata.
- GeneratedJavaAPIsfor retrieval and analysis onNo-SQLdatabase such as HBase.
- CreatedETLjobs to generate and distribute reports fromMySQLdatabase using PentahoDataIntegration.
- Worked on loading data fromUNIXfile system toHDFS
- Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
Confidential, New Jersey
Big Data Analyst
Responsibilities:
- Installed/Configured/Maintained ApacheHadoopclusters for application development andHadoop tools like Hive, Pig, HBase.
- Involved in writing Client side Scripts usingJava Scriptsand Server Side scripts usingJava Beansand usedservletsfor handling teh business.
- CreatedElastic Map Reduce (EMR) clustersand Configured teh Data pipeline wifEMRclusters for scheduling teh task runner.
- DevelopedScalaprograms wif Spark for data inHadoop ecosystem.
- Extensively involved in Installation and configuration ofClouderadistributionHadoop 2, 3, NameNode, Secondary NameNode, JobTracker, TaskTrackers and DataNodes.
- Developed another user basedWeb services (SOAP) through WSDLusing WebLogic application server andJAXBas binding framework to interact wif other components.
- Managed and reviewedHadoop Logfilesas a part of administration for troubleshooting purposes. Communicate and escalate issues appropriately.
- Provisioning ofEc2Instances on both Windows and Linux and worked onAWSRelational Database Services,AWSSecurity Groups and their rules
- Implemented Reporting, Notification services usingAWS API.
- DevelopedMapReducejobs using apache commons components.
- Used Service Oriented Architecture (SOA) basedSOAPandRESTWeb Services(JAX-RS)for integration wif other systems.
- Collected and aggregating large amounts of log data using Apache Flume and staging data inHDFSfor further analysis
- Involved in designing and developing teh application usingJSTL, JSP, Java script, AJAX, HTML, CSS and collection.
- ImplementedAWS EC2, Key Pairs, Security Groups, Auto Scaling, ELB, SQS, and SNS using AWS APIand exposed as teh Restful Web services.
- CreatedHBasetablesto load large sets of structured, semi-structured and unstructured data coming fromUNIX,NoSQLand a variety of portfolios.
- Solved performance issues in Hive and Pig scripts wif understanding of Joins, Group and aggregation and translate to MapReduce jobs.
- DevelopedUDFsinJavaas and when necessary to use inPIGandHIVEqueries.
- Coordinated wif various stakeholders such as teh End Client, DBA Teams, Testing Team and Business Analysts.
- DevelopedJavaWeb Applications usingJSPandServlets,Struts,Hibernate,spring,RestWebServices,SOAP.
- Involved in gathering requirements and developing a project plan.
- Involved in understanding requirements, functional specifications, designing documentations and testing strategies.
- Involved in UI designing, Coding, Database Handling.
- Involved inUnitTestingandBugFixing.
- Worked over teh entireSoftware Development Life Cycle (SDLC)as a part of a team as well as independently.
- WrittenSQLqueriesto query teh database and providing data extracts to users as per request.
Confidential
Java/Scala Developer
Responsibilities:
- Develop Web tier usingSpring MVCFramework.
- Perform database operations on teh consumer portal usingSpringJdbc template.
- Implementeddesign patternsin Scala for teh application.
- Setting upinfrastructureImplementing Configuring ExternalizingHTTPDmod jkmod rewrite.mod proxy JNDI SSL etc.
- Involved in converting Hive/SQL queries into Spark transformations using Spark RDD, Scala.
- ImplementedRestfulservices in Spring.
- Serialize and de-serialize objects usingPlayJson library.
- Developing traits and case classes etc in scala.
- Develop quality code adhering to Scala coding Standards and best practices.
- Writing complex Sql queries.
- Develop GUI usingJQueryJsonandJava script.
- Unit testing Integration testing and bug fixing.
Confidential
Data Analyst
Responsibilities:
- Conducted thorough study to establish teh relationship between various functionalities and worked on teh change booking scenario.
- Developed entity relation diagrams for teh entire change booking functionality using MS Visio
- Executed several complex SQL queries to access, update data from different data bases dat has huge amounts of data
- Visualized teh data using tableau to design data according to customer requirements.
- Communicated teh findings of teh analysis to teh client and teh team.
- Helped development team in understanding teh requirements of teh client.
- Participated in all scrum meetings and addressed teh issues and concerns raised by client immediately
- Developed a New Distribution Capability migration project to enhance usability.
- Teh migration project was implemented for all teh functionalities in Java language using Spring framework.
- Ensured all teh scenarios were passed by writing JUnits for teh code.
- Deployed in teh local environment and tested in SoapUI by writing XML code.
- Committed teh code to Jenkins.
- Increased annual revenue by 10% in 6 months.
- Delivered three major change requirements wifin 2 months by proper planning and executing teh tasks as module lead for theteam.
- Promoted teh project to production wif zero defects by coordinating wif 7 different teams.
Confidential
Test Analyst
Responsibilities:
- Understood and analyzed client requirements to prepare Traceability Matrix, Test Plans, Test Cases and Test Report dat impacted teh project deliverables.
- Performed intensive testing wif different test cases for a particular scenario to assure quality of deliverables
- Identified different bugs and provided a detailed analysis for each bug which helped development team to resolve bugs faster.
- Performed various analysis to gain in-sights for data-driven decision-making on numerous automation projects to identify feasibility and to optimize business processes.
- Completed various levels of functional (using XML service requests and responses), non-functional and assisted thedevelopment team in fixing bugs.