Lead Big Data/hadoop Developer & Architect Resume
3.00/5 (Submit Your Rating)
NJ
SUMMARY
- 9.5+ years of experience in Analysis, Architecture, Design, Development, Testing, maintenance and User training of software applications which includes in Big data, Hadoop & HDFS environment and in Java, J2EE.
- Provide clear direction to application owners concerning misaligned and broken portfolio setup to segregate and align portfolio work streams into enterprise frameworks. Ensure the desired business case objective and scope align with project scope and objectives; work with portfolio business case managers to complete business case change requests, obtain business case workflow approvals, and provide level of estimate demand support.
- Lead senior and junior staff concerning portfolio applications' within the enterprise, roadmap completeness, project management practice processes, and portfolio management components; coach, mentor, develop, and support a Project Analyst's project management capability growth, knowledge depth, confidence, escalation guidance, and general management skills.
- Good experience in developing PIG Latin Scripts, Hive script, customized UDF's in Hive and Pig.
- Expertize with the tools in Hadoop Ecosystem including Pig, Hive, HDFS, MapReduce, Sqoop, Storm, Spark, Scala, Kafka and Oozie.
- Expertise working with NoSQL database including MongoDB, Hbase and DynamoDB.
- Experience with HDFS, MapReduce, Hive, Hbase, Flume and Oozie.
- Good understanding and ability to use SQL, XML, JSON and UNIX.
- Hands - on experience in installing, configuring and monitoring HDFS clusters (on premise & cloud AWS).
- Knowledge in Business Intelligence and Reporting. Preparation of Dashboards using Tableau.
- Knowledge in coding SQL, PL/SQL, Procedures/Functions, Triggers on database (RDBMS) packages like Oracle.
- Experience in collecting business requirements, writing functional requirements and test cases and in creating technical design documents with UML - Use Cases, Class, and Sequence and Collaboration diagrams.
- Expertise in creating Hive Internal/External Tables/Views/Joins and loading different types of data files into Hive like CSV, XML, TXT etc.
- Expertise in Data load management, importing & exporting data using SQOOP & FLUME.
- Experienced on loading and transforming of large sets of semi structured and unstructured data using Pig Latin operations.
- Good knowledge in integration of various data sources like RDBMS, Spreadsheets, Text files, JSON and XML files.
- Extensive experience with Java/J2EE technologies such as Servlets, JSP, EJB, JDBC, JSF, Struts, Spring, SOA, AJAX, XML/XSL, DOJO, Web Services (REST, SOAP), UML, Design Patterns and XML Schemas.
- Have good knowledge and skills implementing Web/Enterprise architectures and in open source frameworks like Struts, Hibernate, Spring Framework (Aspect oriented programming and inversion of control).
- Good knowledge of HTML, XML, XSL, CSS, Power Builder, JavaScript, Tiles, ANT, Maven, JUnit and Log4J.
- Experience with NoSQL databases such as MongoDb, HBase.
TECHNICAL SKILLS
- P ig, Hive, Sqoop, Flume, HBase, Spark, MapReduce, Hadoop Distribution (Horton works,
- Cloudera, Pivotal), Spark, Scala, nIfI, Kafka
- Cloudera, Cent OS, UNIX, Windows, LINUX
- J2SE, J2EE - JSP, Servlets, JNDI, JDBC, JSTL, JMS
- JSP, Servlet, AJAX, JavaScript, JQuery, HTML 5, CSS-3, XML
- JBoss, Web Logic, Tomcat
- Oracle 11g/10g, MySQL, Dynamo DB, MS SQL Server, No-SQL, Cassandra, MongoDB, HBase
- Spring, Struts, Hibernate
- SOAP, REST, Web Inject, Storm
- Eclipse, Net Beans, TOAD, Python
- GIT, CVS, SVN, TFS
- Maven, Ant, JUnit, MRUnit, Log4J
- AWS, Confidential Azure
PROFESSIONAL EXPERIENCE
Confidential, NJ
Lead Big Data/Hadoop Developer & Architect
Responsibilities:
- Involved in Various Stages of Software Development Life Cycle (SDLC) deliverables of the project using AGILE Software development methodology.
- Responsibilities included resource management, client meetings, implementation and design, coordinating off shore teams, budgetary analysis and risk management.
- Followed Agile-Scrum project development methodology for implementation of projects, part of the daily scrum meetings and sprint meetings.
- Cluster management for Hadoop on Cloud, AWS Instances.
- Analyzed requirements and designed data model for Cassandra, Hive from the current relational database in Oracle and Teradata.
- Used MySQL, Dynamo DB and ElastiCache to perform basic database administration.
- Cluster management for Hadoop on Cloud, AWS Instances.
- Developed Spark scripts by using Scala shell commands as per the requirement.
- Loaded the customer profiles data, customer spending data, credit from legacy warehouses onto HDFS using Sqoop.
- Supporting Data analytics team providing various sources data in Hive using Spark SQL.
- Setup Architecture for big data capture, representation, information extraction and fusion.
- Created a Hive aggregator to update the Hive table after running the data profiling job. Analyzed large data sets by running Hive queries.
- Uploaded and processed more than 30 terabytes of data from various structured and unstructured sources into HDFS (AWS cloud) using Sqoop and Flume.
- Creating Hive tables, loading with data and writing Hive queries that will run internally in map reduce way.
- Extracted data from Teradata to HDFS using Sqoop. Analyzed the data by performing Hive queries.
- Implemented Partitioning, Dynamic Partitioning and Bucketing in Hive.
- I have successfully written Spark Streaming application to read streaming twitter data and analyze twitter records in real time using Yardstick framework to measure performance of Apache Ignite Streaming and Apache Spark Streaming.
- Implemented test cases for Spark and Ignite functions using Scala as language.
- Developed Hive queries to process the data and generate the data cubes for visualizing.
- Built reusable Hive UDF libraries for business requirements, which enabled users to use these UDF's in Hive Querying.
- I have successfully written Spark DataFrame application to read from HDFS and analyze 10 million twitter records using Yardstick framework to measure performance of Apache Ignite SQL and Apache Spark DataFrame.
- Used Informatica 9.6 and Sqoop.
- Provide mentorship and guidance to other architects to help them become independent
- Provide review and feedback for existing physical architecture, data architecture and individual code
- Debug and solve issues with Hadoop as on-the-ground subject matter expert. This could include everything from patching components to post-mortem analysis of errors.
- Worked on migrating Map Reduce programs into Spark transformations using Spark and Scala.
- Work on Informatics Power Center, Informatics Power Exchange for Metadata Analysis
Confidential, Fairfax, VA
Software Developer
Responsibilities:
- Prepared the Project plan, daily, weekly status calls with project teams.
- Designed and developed online web application C#, ASP.NET, SQL server, stored procedures pages with web server controls for the presentation layer.
- Implemented validation using validation controls, HTML file, CSS3, JavaScript, JQuery, Ajax, client side validations.
- Created Win Forms and Web Forms for developing the applications using win controls, web server controls and HTML controls.
- Using Entity Framework, connected to relational data by writing LINQ to SQL queries. Worked on client side technologies JQuery, JavaScript.
- Used Master pages, User controls, Custom controls to suit user’s needs.
- Involved in all the phases of the Software Development Life Cycle including System Analysis, Design and Development.
- Used Team Foundation Server (TFS) for Version Control and generated reports using the SQL Server Reporting services (SSRS).
Confidential - Kingsville, TX
Graduate Researcher, Big Data and Cloud Computing
Responsibilities:
- Extended research on Data analysis providing various sources data in Hive using Spark SQL and other scenario.
- Setup Architecture for big data capture, representation, information extraction and fusion.
- Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports to see performance benchmark.
- Provide review and feedback for existing physical architecture, data architecture and individual code.
- Analyzing Hadoop cluster using different Big data analytic tools including Kafka, Pig, Hive and Map Reduce.
- Provide mentorship and guidance to other architects to help them become independent
- Cloud Computing: Successfully deploys & migrate applications to the cloud on Confidential windows azure. Software platform: Confidential visual studio 2013 (C++, pHp, ASP.NET), Team Foundation Server, Eclipse (Luna).
- Cloud Audit: High-Performance Cloud Auditing and Applications.
- Verifies application results by conducting system audits of technologies implemented.
- Cloud Infrastructure Security Challenges & Security Guidance for Critical Areas of Cloud Computing.
- Cloud Computing Security Reference Architecture.
- Cluster management for Hadoop on Cloud, AWS Instances.
- Secure Mobile Cloud Computing and Security Issues.
Confidential
Lead Developer/IT Manager
Responsibilities:
- Understanding the scope of the project and requirement gathering.
- Plan, organize, direct, control and evaluate the operations of information systems and electronic data processing (EDP).
- Maintains organization's effectiveness and efficiency by defining, delivering, and supporting strategic plans for implementing information technologies.
- Verifies application results by conducting system audits of technologies implemented.
- Recommends information technology strategies, policies, and procedures by evaluating organization outcomes; identifying problems; evaluating trends; anticipating requirements.
- Maintains quality service by establishing and enforcing organization standards.
- Maintains professional and technical knowledge by attending educational workshops; reviewing professional publications; establishing personal networks; benchmarking state-of-the-art practices; participating in professional societies.
Confidential
Software Developer/Project Coordinator
Responsibilities:
- Maintained the UI screens using web technologies like HTML, JavaScript, JQuery and CSS.
- Involved in Requirements analysis, design, and development and testing.
- Designed, deployed and tested Multi-tier application using the Java technologies.
- Involved in front end development using JSP, HTML & CSS.
- Documented the changes for future development projects.
- Involved in code deployment, unit testing and bug fixing.
- Prepared design documents for code modified and ticket maintenance.
- Implemented Multithreading concepts in Java classes to avoid deadlocking.
- Used MySQL database to store data and execute SQL queries on the backend.
- Used Apache Ant for the build process.
- Involved in developing JSP for client data presentation and, data validation on the client side with in the forms.
- Used JDBC connections to store and retrieve data from the database.
Confidential
Software Developer
Responsibilities:
- Responsible for telecommunication Network and Infrastructure of Bangladesh which should be intercepted by the National Monitoring Center.
- Maintain Monitoring Center Front-End & Back-End side and contains the devices used to capture intercepts. Examples of such devices are Intercept Management system and devices that intercept voice & packet data.
- Correspondence with local & foreign officials multiple vendor Confidential Network, Confidential, Confidential, Confidential technical personnel’s.
- Maintain dedicated server of National Monitoring Center.
- Perform visual inspection and mechanical maintenance equipment and infrastructure of National Monitoring Center.
Confidential
Software Developer
Responsibilities:
- Designed Web based ERS service tools and operation unit of Distribution for
- Confidential Ltd of the requirements for support of Commercial Division.
- Developing strategic plans & efficient web based solution for electronic product sales & operation department.
- EPSO website specially designed for complete solution of Distribution Unit contain web based ERS Service Operation, Report generating, Service tools & online support providing for Confidential employee.