We provide IT Staff Augmentation Services!

Senior Big Data/aws Engineer Resume

SUMMARY:

  • I have 18 years of Object Oriented software, 15 years of Web, 8 years of Big Data, and 8 years of AWS development experience.
  • For the last 15 years, I have been involved with analysis, architecture, design, and implementation of Distributed, High - Performance, and Scalable Systems, Big Data projects, Services, E-commerce sites, and Shopping Carts through technologies such as Scala, Java, Python, J2EE, OOP (Object Oriented Programming), Functional Programming, Apache Spark, Spark SQL, Spark Streaming, AWS, EMR, DynamoDB, Redshift, Kafka, Hadoop, HDFS, Map/Reduce, Hive, Vertica, Sqoop, Flume, Yarn, Ambari, other Hadoop Ecosystem , Data Lakes, NoSQL, Apache Cassandra and CQL, Data Structures, Java Multithreading, Spring framework, Spring MVC, Spring AOP, Hibernate, JPA, SOAP and REST based Web Services, JAX-RS, Design Patterns , Cloud Computing, Amazon AWS, WebLogic, JBoss, Tomcat, Apache HTTP Server, Data Modeling , SQL, Hibernate, ETL, Business Intelligence, Tableau, MySQL, Oracle, UML, Drupal, Networking, Single Sign On (SSO), SAML, Python, Anaconda, NumPy, Pandas, scikit-learn, Git, SVN, XML, XSL, Junit, Linux, UNIX, Advanced UNIX Shell Scripting, Virtualization, Jenkins, Atlassian (Jira, Fisheye, Crucible, Confluence) , OpenGrok, jQuery, Ajax, JSON, Object Oriented JavaScript, Advance CSS.
  • Furthermore, I am an expert on all aspects of Web site design, E-Commerce, search engine optimization, Shopping Carts, Credit Card Processing, Domain Name Servers, and SSL.
  • I am extremely hands on as far as coding is concerned; I am also a great technical writer.
  • In addition, I have deep understanding of stock market, options, futures, commodities, and money.
  • Furthermore, I read and learn extremely fast (600 wpm)
  • I am persistent, dedicated, meticulous, extremely disciplined and focused, with affection for learning and implementing new technologies, and prototyping.

TECHNOLOGY:

  • EARs, WARs, EJB 2.0, JSP, JSP Customized Library, Servlets, JDBC, JPA, JMS, JAXP, JNDI, JSSE, JTA, JTS . JNI, JavaMail, Multithreading, I18N, Log4J, dom4j.
  • Data Structures, and Collections
  • Java Networking, TCP/IP, UDP, Socket, ServerSocket, DatagramSocket, Proxy, URL, URLConnection.
  • Tomcat 7.0, Jetty, Maven, Ant, Struts 1.3.10, Eclipse 4.5, Scala IDE
  • WebLogic, JBoss Application Server
  • ESB (Enterprise Service Bus), JBossESB
  • Redis, Memcached
  • Scala
  • NumPy, Pandas, Matplotlib, Scikit-learn, sh, os, pymysql, collections, boto3, Anaconda, Jupyter Notebook, PyDev.
  • Hadoop, Setting up Hadoop Clusters, MapReduce, Tez, HDFS, Hive, Flume, Sqoop, Oozie, Apache Ambari, Hue, Zeppelin
  • Spark 2.3, Spark Core, Spark SQL, Spark Streaming, Spark Structured Streaming
  • EMR, Hortonworks Data Platform (HDP), Cloudera Distribution for Hadoop (CDH)
  • Hive, Beeline, Impala, Presto, Vertica, Druid, Phoenix, Drill
  • Kafka 0.10.0
  • Cassandra, Setting up Cassandra Clusters, CQL (Cassandra Query Language)
  • MongoDB
  • Parquet Files, Avro Files
  • Splunk
  • Big Data Predictive Analysis and Modeling
  • REST (JSR-339), JAX-RS, Jersey, CXF
  • Microservices
  • WSDL, SOAP, Axis2, Apache CXF, soapUI
  • JMS, Spring JMS, ActiveMQ, JBossMQ
  • AWS: AWS Architect, EMR, EC2, S3, RDS, Athena, Redshift, IAM, AMI, VPC, ELB, AutoScaling, SNS, SES, SQS, DynamoDB, CloudWatch, ELB, Route 53, Lambda, CloudFormation, aws-cli, boto3.
  • Google App Engine, Google Cloud SQL
  • OOA (Object-Oriented Analysis), OOD (Object-Oriented Design), OOP (Object-Oriented Programming)
  • AOP (Aspect Oriented Programming), DI (Dependency Injection)
  • UML, IBM Rational Rose, Visio, ArgoUML, Design Patterns, J2EE Design Patterns,
  • Portal and Portlets (JSR 168, JSR 286): Liferay Portal.
  • Search Engines: Lucene, Solr.
  • Spring Framework, Spring Web, Spring MVC, Spring AOP, Spring REST, Spring Security, Spring SAML, Struts 2, Hibernate, log4j, Log4jdbc, Tomcat, Apache Jetty, Tiles, Velocity, and ActiveMQ.
  • JUnit, HtmlUnit, JWebUnit
  • JSTL, EL, FileUpload, Configuration, Email, HttpClient, Collections, Logging, C3P0, DBCP. dom4j, Xalan, Freemarker, Quartz (Open Symphony)
  • Relational Databases, SQL, JDBC, ODBC, Toad, Oracle SQL Developer Tool, Oracle Enterprise Manager, Oracle Database Configuration Assistant.
  • PL/SQL, Packages, Procedure, Functions, Triggers
  • Hibernate, JPA
  • JDBC Proxy ( Log4jdbc, P6Spy)
  • Oracle (9i, 10i), MySQL
  • Database Optimization, Normal Forms teratech7.com resumehits.com teasker.com movieweb.com nobleflash.com work-at-home-business.com
  • PHP 4 and 5.
  • Drupal, Drupal Custom Module Development
  • Drupal Commerce module, Ubercart, Views, CCK, Panels, Mini-Panels, Sliders (Ddblock), Video and Embedded Media (SWFTool, FrontMedia, Embedded Media Field).
  • JavaScript, CSS, HTML Related Technologies
  • Object Oriented JavaScript jQuery, jQuery UI, jQuery Form Validation, jQuery Plugins, jQuery custom Plugins, jQuery Mobile, Ajax, JSON.
  • Sublime Text, Brackets, Zen Coding
  • Node.js
  • Handlebars, Mustache
  • CSS 3
  • Responsive Web Design
  • Single-Page Applications
  • HTML5, DOM1, DOM2, Layers
  • SVG (Scalable Vector Graphics) for HTML5
  • Cross browser compatibility issues, Web site and Web page optimization issues.
  • Firebug, and Web Developer (Firefox)
  • Chrome Developer Tools
  • Wireshark (Packet Analyzer), Charles (web proxy)
  • VMware, Oracle VirtualBox, Docker
  • Maven, Ant
  • Apache Subversion (SVN), Git, GitHub, Rational ClearCase, CVS
  • SCM: Branching, Tagging, Rebasing, Merging, Pulling, Pushing
  • Bugzilla, Jira, Rational ClearQuest
  • Atlassian (Jira, Fisheye, Crucible, Confluence, Bamboo, Clover)
  • OpenGrok
  • Jenkins (Hudson), Bamboo
  • Sonar, FindBug
  • Firewalls, DMZ, Content Filtering
  • , Public/Private key, Symmetric Key, SSL, Https.
  • SSH, OpenSSH, SSH Tunneling (Port Forwarding)
  • OpenPGP, PGP, GPG, MD5, SHA-1, SHA-2, AES, 3DES, Nessus Vulnerability Scanner
  • DTD, XML Schema, JDOM, dom4j, Xalan, JAXP, Xerces Parser, XPath 1.0 and 2.0, XSLT 1.0, XMLSpy
  • XQuery 1.0, XLink, XPointer
  • Credit Card Payment Processing, Payment Gateway, Authorize.net, Stripe, PayPal
  • Shopping cart software development, Drupal Commerce, Drupal Ubercart
  • Form Validation and Processing (client and server side)
  • Integration of Credit Card Processing and Payment Gateways.
  • Integration of Check Processing (TeleCheck).
  • Bulk Email Development, Web Crawler Development
  • Web hosting and Virtual Hosting configuration, Portal, Internet, Intranet, and Extranet.
  • Search engine optimization strategies, DNS, Security and Integration
  • RSS, Atom, Blogs, Message Boards, rating, tagging, rich media
  • Camtasia Studio 6, Adobe After Effects CS4
  • Development of static and dynamic online ads, Layer ads, Interstitials, Flash Ads, Rich Media ads, Banner ads, Leaderboards, Skyscrapers, Pop-ups, Scrolling Ads, E-mail ads and campaigns.
  • Ad tracking and ad networks, Google and Bing/Yahoo ads campaign management.
  • Unix (Solaris, and HP-UNIX )
  • Linux
  • Redhat, Fedora, CentOS
  • Debian, Ubuntu, BackTrack
  • SUSE, OpenSUSE
  • GNOME, KDE
  • Advanced Linux Security, Advanced Linux Networking, Intrusion Detection System (IDS), Snort, Tripwire, Wireshark, Squid Proxy
  • Advanced Unix Shell Scripting (Bash)
  • OSX
  • Windows Server 2003 and 2008, Advanced Windows 8 and Windows 10
  • VMware, Oracle VirtualBox
  • Maven, Ant
  • Git, GitHub, Apache Subversion (svn), Rational ClearCase, CVS, SourceSafe
  • Rational ClearQuest, Bugzilla, JIRA
  • Perl, C, C++, Visual C++, Visual Basic
  • Licensed, Foreclosure, Financing, Refinancing, Wholesale, Title, Title Clearance
  • Tax lien, and Tax deed investment

EXPERIENCE:

Senior Big Data/AWS Engineer

Confidential

Environment: Highly Scalable Distributed Applications, ETL pipeline, J2EE 1.8, Scala 2.11, Hadoop, Spark 2.2.0, Spark SQL, Spark Streaming, DataFrame, Scala JSON Parser (Json4s), Tez, Python, Anaconda, NumPy, Pandas, AWS, EMR, AWS Athena, Redshift, S3, EC2, VPC, IAM, SNS, SQS, DynamoDB, Redshift, CloudWatch, ELB, Lambda, AWS-CLI, AWS SDK (Java/Scala), boto3, Kafka 1.1.0, HDFS, Zeppelin, Hive, Hue, Vertica, Presto, Sqoop, Flume, Parquet, Avro, NoSQL, Tableau 10, Scala IDE, Junit, Multithreading, MySQL, Git, Maven, JSON, Jenkins, Scala IDE, Agile development, Atlassian (Jira, Fisheye, Crucible, Confluence).

Responsibilities:

  • Designed, developed, and maintained Confidential (Simple Aggregation Framework).
  • Confidential is a general purpose aggregation engine developed using Spark, Spark SQL, Scala, and Python.
  • Confidential is an internal open source framework which easily accommodates various ETL and aggregation jobs without any need for coding.
  • A JSON formatted Spec Schema dictates the details of the aggregation job to Confidential engine.
  • Integrated Spark SQL into Confidential .
  • Confidential currently aggregates 300 Gig of data per hour on a 500 node EMR cluster for "One Video" platform. "One Video" is one of the 4 video advertising platforms at Confidential .
  • Confidential currently processes data for $600 million dollar of video advertising per annum.
  • Optimized Confidential running on EMR cluster; Optimizations resulted in a 1.5 million dollars saving per year in terms of AWS costs, and failure avoidance.
  • Developed the framework for Real-Time processing of "One Video" data. The real-time framework uses Kafka and Spark, and aggregates data every 5 minutes.
  • Developed unit tests for Confidential engine.
  • Developed various Spark applications for various teams at Confidential .
  • Developed various Python scripts such as Trigger, Schema Generator, Load and Transfer.
  • Maintained Staging and Production EMR clusters.

Senior Big Data/Java Engineer

Confidential

Environment: Highly Scalable Distributed E-commerce Application, AOS ( Confidential Online Store), Red Hat Linux 5, J2EE 1.8, Scala 2.11, Apache Hadoop, Hortonworks Data Platform (HDP), Spark, Spark SQL, Spark Streaming, Spark Windowed Streaming, Kafka 0.10.0, Apache Ambari, Map/Reduce, HDFS, Hive, Sqoop, Flume, Parquet, NoSQL, Apache Cassandra and CQL, MongoDB, Apache Drill, Spring 4.3, Hibernate 5.0.2, Spring MVC, Apache Tomcat, Servlet, log4j, Eclipse, Junit, Multithreading, MySQL, Oracle, Hibernate, Oracle SQL Developer, SVN, Maven, CSS3, JavaScript, jQuery, Ajax, JSON, Jenkins, OpenGrok, JDBC proxy (Log4jdbc), Eclipse, Agile development, Atlassian (Jira, Fisheye, Crucible, Confluence), Splunk.

Responsibilities:

  • Participated in 2 major Community team migrations. The first migration consisted of moving from legacy technology to Tomcat, Spring, Spring MVC, JSON. The second migration consisted of incorporating Hadoop, Spark, Kafka, and Cassandra.
  • Developed Top Products service. Top Product service keeps track of top 100 products in each AOS category in a sliding window of one hour. The service is based on Spark streaming, Kafka, and Cassandra.
  • Developed Product Analysis platform. Product Analysis platform uses Spark to aggregate all AOS product sales data for BI analysis. The source and sink for the data were both HDFS as part of AOS Data Lake.
  • Developed AOS Product Trending service using Spark Streaming. The input data stream was provided by Kafka. The results were output to Kafka for downstream processes, and to Cassandra table to be used by other services.
  • Administered Hortonworks Hadoop cluster.
  • Implemented new features such as Confidential Id Delete feature for Community.
  • Performed Hive, and Cassandra data modeling.
  • Provided production support for Community, and Merchandising pages and Services.
  • Implemented solutions for a large number of bugs, and production support cases.
  • Created and administered a Cassandra cluster consisting of 200 nodes expanding 2 data centers.
  • Optimized Community Services methods and their related SQL queries resulting in 80 percent improvement in the performance of optimized methods. Log4jdbc was utilized to log and profile SQL queries used by the methods.
  • Developed Java Applications and CQL queries to perform CRUD operations on Cassandra cluster.
  • Performed routine maintenance on Hortonworks Hadoop cluster such as adding/removing nodes, adding/removing services.
  • Performed Crucible code reviews for Community and Merchandising Support teams.

J2EE/Web Architect, Hadoop Developer

Confidential

Environment: E-commerce Application, J2EE 1.6, Spring 3.0.6, Hibernate 3.6.0, Spring MVC, REST API Development, Apache Tomcat, AWS, EC2, S3, Apache Cassandra, CQL (Cassandra Query Language), Apache Hadoop, Hortonworks Data Platform (HDP), Cloudera Distribution for Hadoop (CDH), Hive, Sqoop, Flume, Apache HTTP Server, Servlet, JMS, ActiveMQ, HtmlUnit, Multithreading, MySQL, Oracle, Hibernate, Oracle SQL Developer, Git, GitHub, Maven, HTML5, CSS3, JavaScript, jQuery, jQuery UI, Ajax, JSON, Jenkins, OpenGrok, Data Mining, Eclipse, Sublime Text, Zen Coding, Web Services, REST, Jersey, Agile development, Bugzilla, Drupal ( Drupal Custom Module , Drupal Commerce , Panels, Pagination, Views, Slide Shows).

Responsibilities:

  • Designed and developed highly scalable distributed web applications for our clients based on J2EE, Hibernate, Spring, REST web services, Oracle and MySQL, NoSQL, and MVC pattern.
  • Designed and developed REST web services using Jersey framework.
  • Developed API's for various clients.
  • Maintained company's infrastructure and EC2 instances on AWS.
  • Developed the "Mailing List/Discussion Groups Data Mining" application. Given certain regular patterns, this application will return the daily occurrence of these regular patterns in thousands of mailing lists.
  • Implemented all the REST based service endpoints consumed by DCMD utilizing JAX-RS (Jersey).
  • Implemented SAML 2.0 based Single Sign through Spring SAML; only SAML Service Provider (SP) was required. IDP side was setup for testing.
  • Created and initially administered a Cassandra cluster consisting of 100 nodes for HSBC.
  • Developed advanced Shell scripts for test automation.
  • Implemented complete E-Commerce stores for clients.
  • Installed a Hadoop cluster consisting of 80 nodes, including Hadoop ecosystem tools, on AWS. Name Node, Secondary Node, and Ecosystem tools were installed on separate EC2 instances.
  • Created Hive queries to analyze click stream data; results of analysis were used to improve conversion rates.
  • Handled SSL process and implementation for clients.
  • Created the Word2Html Java application which processed Word files into complete Drupal Web sites. Each Word file was not only converted and sanitized, but also paginated and tagged using online services. Thus very long Word files were converted to hundreds of Html Pages in order to allow more ad impressions for the site. Word2Html was hosted on AWS.
  • Developed various Drupal custom modules.
  • Created Email Marketing campaigns through MailChimp, and combination of Sendy and Amazon SES (Simple Email Service).
  • Developed Java based crawlers to automate the login, download, and processing of reports from affiliates such as HSBC bank, and Bank of America.
  • Prototyped systems for client companies.
  • Identified and resolved memory leaks and performance bottleneck utilizing JProfiler profiler.

Java/Web Tech Lead

Confidential

Environment: E-commerce Application, Linux, J2EE 1.6, Cloud Computing (AWS), JBoss 5.0.0, Web Services, Apache Axis2 1.2, SOAP, Spring, Spring JDBC Template, Struts 2, JSP, JSTL, Servlet, JDBC, Ajax, HtmlUnit, Freemarker, Jasypt, Multithreading, Open Symphony Quartz, LDAP (OpenDS), MySQL, Hibernate, CSS, Object Oriented JavaScript, jQuery, jQuery UI, SVN, Hudson (For Continuous Integration), Bugzilla, SSH.

Responsibilities:

  • Lead a team of 6 developers to develop and launch the Job Board Marketing (JBM) Service. JBM is a Web application and was implemented using MVC architectural pattern. The application uses Spring, Struts2, and Hibernate through JPA.
  • Designed and implemented Best Resume's BI (Business Intelligence) dashboard which used Ajax, Spring, Hibernate, Struts2 deployed on Amazon EC2 servers.
  • Administered Oracle (9i) Server and Cluster.
  • Evaluated new technologies. Added jQuery, and jQuery UI as new technologies to be used in different projects.
  • Implemented the Bulk Email Marketing module utilizing JavaMail, and FreeMarker templates. The Bulk Email Marketing sends out millions of direct marketing emails per year.
  • Implemented the required Web crawlers to login into user accounts on job board sites such and Monster, Yahoo Hotjobs, and Dice, and perform the necessary tasks on the users’ accounts automatically. The crawlers were implemented using HtmlUnit.
  • Automated the scheduling and triggering of crawlers actions by utilizing the Open Symphony Quartz.

Senior J2EE Developer

Confidential

Environment: Internet Applications, UNIX (Solaris), J2SE, J2EE, WebLogic Application Server 7.0, Web Services, ERP, UML, Rational Rose, Design Patterns, EJB, Message Driven Beans, JSP, Servlet, JMS, JDBC, JNDI, Multithreading, JDOM, WebLogic Portal 7.0, JProfiler, ClearCase, Oracle9 i , SQL, PL/SQL, Ant, CSS, JavaScript, SOAP, SOAPTest, Apache Axis.

Hire Now