We provide IT Staff Augmentation Services!

Sr. Bigdata Architect Resume

2.00/5 (Submit Your Rating)

South Portland, ME

SUMMARY

  • 9+ years of experience in SDLC with key emphasis on the trending Big Data Technologies - Spark, Scala, Spark Mlib, Hadoop, Tableau, Cassandra, Java, J2EE.
  • Architect, design & develop Big Data Solutions practice including set up Big Data roadmap, build supporting infrastructure and team to provide Big Data.
  • Built streaming applications using SPARK Streaming.
  • Knowledge onbig-datadatabase HBase and NoSQL databases Mongo DB and Cassandra.
  • Expertise in Java Script, JavaScript MVC patterns, Object Oriented JavaScript Design Patterns and AJAX.
  • Experience includes Requirements Gathering, Design, Development, Integration, Documentation, Testing and Build.
  • Experience in working with MapReduce programs, Pig scripts and Hive commands to deliver the best results.
  • Good Knowledge in Amazon Web Service (AWS) concepts like EMR and EC2 web services which provides fast and efficient processing of TeradataBigDataAnalytics.
  • Experienced in collection of LogDataand JSONdatainto HDFS using Flume and processed thedatausing Hive/Pig.
  • Experience with Java web framework technologies like Struts2, Camel and Spring Batch.
  • Having good experience in Hadoop framework and related technologies like HDFS, MapReduce, Pig, Hive, HBase, Sqoop and Oozie
  • Expertise inDataDevelopment in Hortonworks HDP platform & Hadoop ecosystem tools like Hadoop, HDFS, Spark, Zeppelin, Hive, HBase, SQOOP, flume, Atlas, SOLR, Pig, Falcon, Oozie, Hue, Tez, Apache NiFi, Kafka.
  • Strong knowledge and experience in Object Oriented Programming using Java.
  • Extensively worked on development and optimization of MapReduce programs, PIG scripts and HIVE queries to create structureddatafordatamining.
  • Expertise in developing the presentation layer components like HTML, CSS, JavaScript, JQuery, XML, JSON, AJAX and D3.
  • Managed the project based on Agile-Scrum Methods.
  • Worked on Bootstrap, Angular JS and Node JS, knockout, ember.js, Java Persistence Architecture (JPA).
  • Good experience/exposure to scripting languages like UNIX and jobs scheduler like CRON jobs
  • Experience on Machine Learning and data analytics on Big data set.
  • Strong experience in developing Enterprise and Web applications on n-tier architecture using Java/J2EE based technologies such as Servlets, JSP, Spring, Hibernate, Struts, EJBs, Web Services, XML, JPA, JMS, JNDI and JDBC.
  • Experienced in managing and reviewing the Hadoop log files.
  • Developed applications based on Model-View-Controller (MVC).
  • Experienced with programming language such as C, C++, Xpath, Core Java and JavaScript.
  • Extensive experience in building and deploying applications on Web/Application Servers like Web logic, Web sphere, and Tomcat.
  • Expert in AmazonEMR, Spark, Kinesis, S3, Boto3, Bean Stalk, ECS, Cloud watch, Lambda, ELB, VPC, Elastic Cache, Dynamo DB, Redshit, RDS, Aethna, Zeppelin & Airflow.
  • Expertise in core Java, J2EE, Multithreading, JDBC, Hibernate, Spring, Shell Scripting and proficient in using Java API's for application development.
  • Good at problem-solving skills to identify areas of improvement and incorporating best practices for delivering quality deliverables.
  • Have good experience, excellent communication and interpersonal skills which contribute to timely completion of project deliverable well ahead of schedule.

TECHNICAL SKILLS

Languages: Java, J2EE, PL/SQL, Pig Latin, HQL, R, Python, XPath, Spark

Hadoop/Big Data: Map Reduce, HDFS, Hive, Pig, HBase, Zookeeper, SQOOP, Oozie, Flume, Scala, Akka, Kafka, Storm, Mongo DB

Java/J2EE Technologies.: JDBC, Java Script, JSP, Servlets, JQuery

No SQL Databases: Cassandra, mongo DB

Web Technologies: HTML, DHTML, XML, XHTML, JavaScript, CSS, XSLT, AWS, Dynamo DB

Web/Application servers: Apache Tomcat6.0/7.0/8.0, JBoss

Frameworks: MVC, Struts, Spring, Hibernate.

AWS: AWS, EC2, S3, SQS

Operating Systems: UNIX, Ubuntu Linux and Windows, Centos, Sun Solaris.

Network protocols: TCP/IP fundamentals, LAN and WAN.

Databases: Oracle 12c/11g/10g/9i, Microsoft Access, MS SQL

PROFESSIONAL EXPERIENCE

Confidential, South Portland, ME

Sr. Bigdata Architect

Responsibilities:

  • Implemented solutions for ingesting data from various sources and processing the Data-at-Rest utilizing Big Data technologies such asHadoop, Map Reduce Frameworks, HBase, Hive.
  • Performed data analysis, feature selection, feature extraction using Apache Spark Machine Learning streaming libraries in Python.
  • Loaded and transformed large sets of structured, semi structured and unstructured data using Hadoop/Big Data concepts.
  • Experience in AWS, implementing solutions using services like (EC2, S3, RDS, Redshift, VPC).
  • Extensively development experience in different IDE like Eclipse, Net Beans and IntelliJ.
  • Worked as a Hadoop consultant on (Map Reduce/Pig/HIVE/SQOOP).
  • Worked using Apache Hadoop ecosystem components like HDFS, Hive, SQOOP, Pig, and Map Reduce.
  • Good exposure to Github and Jenkins.
  • Exposed to Agile environment and familiar with tools like JIRA, Confluence.
  • Provided recommendations to machine learning group about customer roadmap.
  • Sound knowledge in Agile methodology- SCRUM, Rational Tools.
  • Lead architecture and design of data processing, warehousing and analytics initiatives.
  • Exploring with theSparkfor improving the performance and optimization of the existing algorithms in Hadoop usingSparkContext,Spark-SQL, Data Frame, Pair RDD's,SparkYARN.
  • Used ApacheNififor ingestion of data from the IBM MQ's (Messages Queue).
  • Identify query duplication, complexity and dependency to minimize migration effortsTechnology stack: Oracle, Hortonworks HDP cluster, Attunity Visibility, Cloudera Navigator Optimizer, AWS Cloud and Dynamo DB.
  • Developed custom code to read the messages of the IBM MQ and to dump them onto theNifiQueues.
  • Creation of a User Interface to search and/or view content within the cluster by using solar cloud.
  • As a POC, usedSparkfor data transformation of larger data sets.
  • Worked on setting up and configuring AWS'sEMRClusters and Used Amazon IAM to grant fine-grained access toAWSresources to users
  • Worked on Sequence files, RC files, Map side joins, bucketing, partitioning for Hive performance enhancement and storage improvement.
  • Developed dynamic JSP pages with Struts. Used built-in/custom Interceptors and Validators of Struts.
  • Implemented new role based security system for OASIS application on state portal database.
  • Enable and configure Hadoop services such as HDFS, YARN, Hive, Ranger, Hbase, Kafka, Sqoop, Zeppeline Notebook and Spark/Spark2.
  • Worked on Spark, Scala, Python, Storm Impala.
  • Extensive experience in Spark Streaming (version 1.5.2) through core Spark API running Scala, Java to transform raw data from several data sources into forming baseline data.
  • Creating dashboard on Tableau and Elastic search with Kibana.
  • Hands on expertise in running the SPARK & SPARK SQL.
  • Experienced in analyzing and Optimizing RDD's by controlling partitions for the givendata.
  • Worked on MapR Hadoop platform to implement BigData solutions using Hive, Map Reduce, shell scripting, and java technologies.
  • Struts (MVC) is used for implementation of business model logic.
  • Worked with Struts MVC objects like Action Servlet, Controllers, validators, Web Application Context, Handler Mapping, Message Resource Bundles, and Form Controller.
  • Created heat map/donut pie/histogram and other kinds of reports and created dashboards out of them.
  • Evaluate deep learning algorithms for text summarization using Python, Keras, TensorFlow and Theano on cloudera Hadoop Sstem
  • Deployed and managed Application on Tomcat server.
  • Experienced in queryingdatausing Spark SQL on top of Spark engine.
  • Experience in managing and monitoring Hadoop cluster using Cloudera Manager.
  • Developed different kinds of interactive graphs in R studios.
  • Created own shiny-server on Linux Centos OS and deployed reports on server.
  • Created ER diagram for Data Modeling.

Environment: Big Data, JDBC, NOSQL, Spark, YARN, HIVE, Pig, Scala, Nifi, intellij, AWS EMR, Python, Hadoop, Redshift.

Confidential, Iowa City, IA

Bigdata Architect

Responsibilities:

  • Single View Of Product - Developed scripts using SQOOP, SCP & Hive to consolidate PCM & PSSA attributes of all product sold at Lowe's. Oozie coordinator is used for scheduling.
  • Consolidation of Allied BU Sales, Inventory, customer, GL & otherdata- DevelopeddataIngestion pipeline using SQOOP & Falcon. Developed scripts using Bash, Spark, Hive, Pig.Data Visualization using MSTR VI.
  • Data modeling, schema designing for no-sql (HBase) and impala tables.
  • Worked on Spark Storm, Apache and Apex and python.
  • Implemented the Machine learning algorithms using Spark with Python
  • Single View Of Product - Developed scripts using SQOOP, SCP & Hive to consolidate PCM & PSSA attributes of all product sold at Lowe's. Oozie coordinator is used for scheduling.
  • Developed the application by using the Spring MVC framework.
  • Delivery experience on major Hadoop ecosystem Components such as Pig, Hive, Spark Kafka, Elastic Search &HBase and monitoring with Cloudera Manager.
  • Collection framework used to transfer objects between the different layers of the application.
  • Involved in installing EMR clusters on AWS.
  • Implementation of projects using Agile SCRUM product development strategy.
  • Swings being used to customized the specific tools for generic purpose.
  • Documented application for its functionality and its enhanced features.
  • Developed a shell script to create staging, landing and Semantic tables with the same schema like the source.
  • Developed HiveQL scripts for performing transformation logic and also loading thedatafrom staging zone to landing zone and Semantic zone.
  • Responsible for Debug, Optimization of Hive Scripts.
  • Involved in installing EMR clusters on AWS.
  • Used Git for version controller.
  • Use Spark API for Machine learning. Translate a predictive model from SAS code to Spark
  • Swings being used to customized the specific tools for generic purpose.
  • Documented application for its functionality and its enhanced features
  • IRIS - Implemented Alteryx workflow to load IRISdatafrom Cassandra DB in MS Azure to our analytics cluster & used SPARK scripts fordataclean up & formatting. Built Hive scripts to preparedatafor analytics.
  • UsedSparkAPI over Cloudera Hadoop YARN to perform analytics on data in Hive.
  • Develop DAO design pattern for hiding the access to data source objects.
  • Worked directly with product owners to get requirement and implemented it.
  • Actively participated in planning sessions for the Sprints, effort estimations, Backlog refinements and dividing features into User Stories and Tasks.
  • Involve in implementation of REST and SOAP based web services.
  • Worked on improvising the performance of the application.
  • Worked on Spark Storm, Apache and Apex and python.
  • Used AWSDataPipeline to schedule an AmazonEMRcluster to clean and process web server logs stored in Amazon S3 bucket.
  • Sales Forecast & USHI metrics - Implemented Hive scripts for implementing sales metrics for reporting & forecasting. Oozie is used for scheduling.
  • Developed Shell & Hive scripts for consolidating BrandView pricingdata. Oozie is used for scheduling.
  • Spring IOC being used to inject the parameter values for the Dynamic parameters.
  • Actively involved in code review and bug fixing for improving the performance.

Environment: Apache Hadoop, HDFS, Hive, Map Reduce, Impala, Cloudera, Pig, SQOOP, Kafka, Spark, Apache Cassandra, Oozie, Impala, Cloudera, Zookeeper, MySQL, Eclipse, PL/SQL and Python.

Confidential, Minnesota, MN

Sr. Java/J2EE Developer

Responsibilities:

  • Install & Configure Oracle Fusion Middleware products on Linux environments.
  • Work involving Oracle FMW products (SOA Suite, OSB, ODI, Web Center, and WebLogic).
  • The interfaces are built using Apache Camel framework and JMS.
  • Proficient in TDD (Test Driven Development) by writing test cases using Jasmine, J-Unit Framework Working as a Full Stackdeveloperusing JavaScript frameworks Angular.JS and NodeJS.
  • Used Spring Core Annotations for Dependency Injection and used Apache Camel to integrate Spring framework.
  • Used Quality assurance of client/server and web based applications using Quick Test Professional QTP.
  • Integrated Automated functional tests (Groovy) with Continuous-Integration in Jenkins.
  • Responsible for creating efficient design and developing User Interaction screens using HTML5, CSS3, JavaScript, JQuery, AJAX, Angular JS, and JSON Used jQuery, Ajax for service calls on pages to interact with the servers.
  • Worked on importing impala to Python.
  • Designed and developed business applications using JSP, Servlet, JAVA, J2EE, Threads, Socket programming, EJB, XML, JNDI, Hibernate and JDBC technologies on Windows and UNIX platform.
  • Implemented XML-based communications protocol using JAXB.
  • Used Maven as the build tool, GIT for version control, Jenkins for Continuous Integration and JIRA as a defect tracking tool.
  • Used DevOps Methodologies for reliable and faster software delivery pipeline
  • Developed REST API for test server for Express Router middleware based API's with Mongo DB integration. Strong Experience in developing user interfaces with HTML, DHTML, JSTL, XSD, XML and CSS. Worked with scripting languages like JavaScript, AJAX and JQuery.
  • Planning and setting up of Continuous Integration for various properties on Jenkins with Commit, Component, Assembly, Deploy and Smoke jobs.
  • Built and DeployedJava/J2EE to a web application server in an Agile continuous integration environment and also automated the whole process.
  • Experience working withbigdataand real time/near real time analytics andbigdataplatforms like Hadoop, Spark using programming languages like Scala and Java.
  • Wrote programs in Scala using Spark and worked on migrating MapReduce programs into Spark using Scala.
  • Maven tool has been used for the build and deploy procedure scripts in UNIX, Linux and Windows environment using Hudson.
  • Responsible for creation of Source to Target mapping document from source fields to destination fields mapping.
  • Developed a shell script to create staging, landing and Semantic tables with the same schema like the source.
  • Extensively involved in the development of backend logic or data access logic using Hibernate and thus creating the object relational mapping with theJavabeans in the application.
  • Worked on performance tuning of Web Agents, Policy Servers, Policy Stores and User Stores to meet and maintain operational requirements (process, thread, connection, cache).

Environment: Java, J2EE, Java SE 6, UML, JSP 2.1, Hadoop 1x, Hive, Pig, HBASE, JSTL 1.2, Servlets 2.5, Spring MVC, Hibernate, JSON, Restful Web services, jQuery, UNIX, AJAX, Angular Js, JAXB, IRAD, Web Sphere 7.0, Eclipse Kepler-Maven, Serena Dimensions, Unix, JUnit, DB2, Oracle.

Confidential

Sr. Java/J2EE Developer

Responsibilities:

  • Implemented Object-relation mapping in the persistence layer using hibernate frame work in conjunction with Spring Aspect Oriented Programming (AOP) functionality.
  • Developed application framework using struts with design principles of the J2EE using Business Delegate, Service Locator, Session Facade, Domain object and DAO patterns and developed Stateless Session Bean to Achieve Session façade Design Pattern.
  • Developed Stored Procedures and triggers using PL/SQL in order to calculate and update the tables to implement business logic.
  • Developed SQL queries and Stored Procedures using PL/SQL to retrieve and insert into multiple database schemas.
  • Help Devops teams configuring servers by building cook books to install and configure tomcat.
  • Developed the XML Schema and Web services for the data maintenance and structures Wrote test cases in JUnit for unit testing of classes.
  • Used DOM and DOM Functions using Firefox and IE Developer Tool bar for IE.
  • Used JSP, HTML, Java Script, Angular JS and CSS3 for content layout and presentation.
  • Did core Java coding using JDK 1.3, Eclipse Integrated Development Environment (IDE), clear case, and ANT.
  • Developing User Interface Screens using Spring MVC, to enable customers obtain auto finance. Extensive experience in developing various web based applications using Hibernate 3.0 and Spring frameworks.
  • Developed Spring REST Exception Mappers.
  • Developed Authentication layer using Spring Interceptors.
  • Used Log4J to print the logging, debugging, warning, info on the server console.
  • Build test cases using JUnit and carried out unit testing.
  • Developed Spring REST Exception Mappers.
  • Developed Stored Procedures and triggers using PL/SQL in order to calculate and update the tables to implement business logic.
  • Responsible for deployment of application in the Integration/Functional Environment, providing necessary assistance to UA Tester.
  • Developed application framework using struts with design principles of the J2EE using Business Delegate, Service Locator, Session Facade, Domain object and DAO patterns and developed Stateless Session Bean to Achieve Session façade Design Pattern.
  • Developed Stored Procedures and triggers using PL/SQL in order to calculate and update the tables to implement business logic.

Environment: Java, XML, HTML, JavaScript, JDBC, UNIX, CSS, SQL, PL/SQL, XML, Web MVC, Eclipse, Ajax, JQuery, Spring with Hibernate, Active MQ, Jasper Reports, Ant as build tool and My SQL and Apache Tomcat.

We'd love your feedback!