OBJECTIVE:

Around 10 years of IT experience in Software Development with proficiency in designing, development of Enterprise - based, Web-based applications and Big Data Engineering Technologies. I am seeking a position as a Spark Developer.

SUMMARY:

• Experienced in implementing Scala’s mutable, immutable types and control structures.
• Experienced in leveraging Scala’s Collections such as: Arrays, Sets, Maps, List and Tuples.
• Proficient in writing functions using Scala language and expert knowledge in Scala’s access modifier types.
• Knowledge of traits and how they are constructed in Scala, abstract and concrete classes.
• Proficient in implementing error handling, break and continue functionality in Scala.
• Expert knowledge of Spark Architecture, RDD and Dataframe abstractions using Spark SQL to aggregation and execute queries involving various datasets.
• Proficient in using Maven to create software and their dependencies.
• Expert knowledge in importing Spark libraries into projects, such as: SparkConf and SparkContext.
• Expert knowledge in YARN deployments involving the Client and Cluster mode.
• Recent project experience with Confidential includes working knowledge of Hadoop ecosystem. Recent project experience with Confidential includes working knowledge of Hadoop ecosystem: HDFS and data processing in MapReduce, YARN architecture, Hadoop Configuration, Pig, Hive and HiveQL, Apache Flume, Sqoop, Oozie, NoSQL databases, Apache HBase, Zookeeper, Ambari installation, EC2 server installation for Hadoop/Hortonworks/Cloudera, EMR and S3 Storage configuration. Proficient in creation, configuration of HDFS and installation of the abstraction layers.
• Proficiency in creation, configuration of HDFS and installation of the abstraction layers, Apache Spark pipeline and streaming Apps and concepts. Well-versed in both Client and Server-side Programming using Java Technologies and deploying applications into production server and maintenance. Technically skilled and working knowledge of IDE Tools like Eclipse, NetBeans, Rational Application Developer (RAD) for Java/J2EE application development, writing LINUX/UNIX Scripts, Developing Shell scripting and on LINUX/UNIX commands. Proficient in Node.js installation and configuration enabling front and backend development. Experienced in DataStage Transformation for batch processing, Agile Requirements Designer (ARD), Spring Tools Suite, IBM Integration Bus development (IIB) and MQ Administration.
• Experienced in design, transformation of datasets using WebSphere Message Broker and MQ. Experienced in writing ESQL code for various nodes in IBM WebSphere Message Broker. Proficient in WebSphere Message Broker SOAP Nodes Web Service handling, HTTP Nodes, Error Handling, Java Compute Node, HTTPS and REST protocols.
• Experienced in design, extraction and loading of datasets using DataStage Data Warehousing and Data Mart technology.
• Experienced in design, mapping and transformation of functional maps using IBM Transformation Extender (ITX)
• Experienced in using Business Intelligence Tools such as Tableau and BI.
• Working knowledge of OLAP (Online Analytical Processing) technology and Dimensional Modeling used for most commercial OLAP products.
• Architected PM activities to ensure that resources, risk recognition, and planning stay aligned with the solution goals.
• Working knowledge of star schema and the snowflake schema dimension tables to describe data aggregated in a fact Table.
• Excellent communication and interface skills.

PROFESSIONAL EXPERIENCE:

Confidential

Big Data and Middleware Engineer

Responsibilities:

• Experienced with Hadoop abstraction layers in the Hadoop ecosystem such as: Apache Hadoop, Hive, Pig, YARN and MapReduce, Sqoop, Kafka, Flume, HBase, Ambari Dashboard, Ganglia.
• Experienced with data ingestion techniques using Sqoop to extract data from RDBMS to HDFS, Hive and HBase.
• Experienced with data ingestion techniques using Flume from the source, to Flume Agent and HDFS sink.
• Experienced with building virtual machines and installation of Hadoop software tools on top of Hadoop and APIs.
• Experienced with Kafka Message Broker implementation.
• Familiar with NoSQL Databases such as Cassandra, MongoDB, HBase and Giraph.
• Familiar with Graph Databases such as Unicorn, FlockDB, Neo4j and Apache Giraph.
• Familiar with Document Databases such as MongoDB, Couchone and TerraStore.
• Experienced with HDFS Block and File storage levels.
• Experienced with MapReduce Frameworks such as: Input, Mapper Combiner, Sort and Shuffle Partitioner, Reducer and Output Stages.
• Experienced with Spark computation engine.
• Experienced with Spark Abstraction RDD transformation, lazy evaluation, DAG and Action.
• Experienced with functional programing in the Spark shell.
• Experienced with Scala pure object oriented and Java programming.
• Experienced in maintaining a centralized configuration, naming, distributed synchronization using group coordination service such as Zookeeper.
• Proficient in enabling Apache Hadoop with High Availability (Journal Node and Zookeeper).
• Proficient in decommissioning services in the Hadoop ecosystem.
• Experienced in Cloud services such as AWS, Hortonworks and EMR.
• Experienced in HDFS commands, User Management and troubleshooting.
• Knowledge of processing layers, Job Tracker and Task Trackers.
• Knowledge of Hadoop execution modes such as: Standalone, Pseudo mode and Fully distributed mode.
• Deep understanding of Lambda and Kappa Architecture.
• Experienced in utilizing data visualization tools such as Tableau and BI.
• Installed and configured WebSphere MQ 7.5,8.0, IBM Message Broker v9.0, v10.
• Utilized Compute Nodes in IBM Integration Bus (IIB) Technology; implemented business logic and transformations using ESQL and Java Language in the development of message flows.
• Developed WebSphere Transformation Extender (WTX) Type Trees to be used in complex mapping.
• Integrated WTX plugins with IBM Integration Bus and configured the flow to run seamlessly.
• Implemented MQMFE/MFT file transfer in MQ Explorer.
• Configured MQMFT scheduled file transfers as needed.
• Developed interfaces using middleware tools: IIB, DataStage, ITX and Sterling MFT.
• Participated in Scrum and retrospective meetings.
• Implemented Agile Methodology and Kankan.
• Adhered to software development standards in maintaining clean code after development.
• Participated in knowledge transfer sessions as needed.
• Experienced in using business requirements to create Test Suites and Test cases using Agile Designer (ARD).
• Developed code using API and UI development including PCF, spring Tool Suite, maven and Artifactory.
• Implemented and utilized Test Automation tools to generate Test Cases, including Jenkins, Sonar, Java CAF and CA Agile Designer.

Technologies Used: Hadoop, Spark, IIB, MQ, ITX, DataStage

Confidential

Software Engineer

Responsibilities:

• Installed and configured of WebSphere MQ 7.5,8.0, IBM Message Broker V9.0
• Used various nodes, ESQL and Java Language in the development of the flows
• Developed WTX Type Trees for Confidential Migration projects.
• Integrated WTX Maps to communicate with IBM Integration Bus using Message Sets or DFDL Language Models.
• Implemented MQMFE/MFT file transfer in MQ Explorer.
• Configured MQMFT scheduled file transfers as needed.
• Developed interfaces using middleware tools: IIB, DataStage, ITX and Sterling MFT.

Technologies Used: ITX, IIB, MQ

Confidential

Software Developer

Responsibilities:

• Installed and configured of WebSphere MQ 7.5, IBM Message Broker V 9.0; utilized various nodes and ESQL in the development of the flows.
• Developed error handling message flows to handle various types of errors and notifying the users.
• Worked with Operational Decision Manufacturer (ODM) team in integration with IIB.
• Created message sets for XML data with client XSD’s.
• Developed rules and ESQL for message content enrichment and transformation.
• Performed root cause analysis and failures in production environment.
• Assisted in defining the message structure/format and business process rules for each interface.
• Developed message flows that use Flow-Order, Try-Catch, Throw, Trace nodes for routing messages properly and exception handling.
• Proficient in Business process Management (BPM), Model creation and Playback implementation of User interfaces, Integrations, start Reporting / Monitoring and creation of Events / Messaging, finish Reporting / Monitoring.
• Developed and implemented WebSphere Transformation Extender (WTX) Type Tree designs based on the business requirements.
• Worked with migration type on complex migration projects integrating (IIB) with WebSphere Transformation Extender (WTX)

Technologies Used: ITX, IIB, MQ

Confidential

Software Engineer

Responsibilities:

• Designed and implemented orthographic projections using AutoCAD software.
• Worked on capital projects and cost estimation projects using Microsoft Access and Excel database designs.
• Arranged interlock meetings with vendors.
• Performed various analytical tests involving coal, water, fuel and Ash mineral samples.
• Installation and configuration of WebSphere MQ 7.5, IBM Message Broker V9.0
• Extensive usage of various nodes, ESQL and Java Language in the development of the flows

Technologies Used: IIB,ITX

Confidential

Scientist/Chemical Engineer

Responsibilities:

• Prepared solutions for laboratory usage, analyzed and recorded test results as required.
• Maintained testing equipment in suitable working order and performed safety checks regularly. Assisted in writing test results and reviewed them as needed.
• Provided guidance to laboratory technicians and performed other lab-related duties as assigned; such as glassware and laboratory equipment.
• Tested and analyzed products to verify stability, chemical reactions and other characteristics; conducted compound tests on lab samples according to established procedures.
• Setup and maintained apparatus used in lab testing; reported on defective equipment or unusual occurrences.
• Assisted in the implementation of new testing methods and techniques-R & D studies and formulation of new products.