Data Engineer Resume
Princeton, NJ
PROFESSIONAL SUMMARY:
- Experience working in multiple roles of software development life cycle including analysis, design, development. Countable experience in data engineering, client/server architecture along with web and windows based applications.
- Good at using Google Cloud Platform (GCP) including Docker Containers, Container Builder, Container Engine (Kubernetes), Network Load Balancer (Ingress & Node Port) with Certificates (HTTPS), Cloud Storage, Data Store (NoSQL), Data Proc (managed Hadoop), and Cloud Pub/Sub (managed Queues), Big Table (managed HBase) etc.
- Mastered Hadoop core components including HDFS, Map Reduce, YARN, thorough understanding of MapReduce features including various types of joins as well as writing scripts to analyze the data in Pig Latin.
- Have designed and developed Hive databases to organize data and used various techniques including partitioning, bucketing to store data appropriately
- Have written Data Pipelines using Lambda architecture for batch as well stream processing using Kafka, Spark, Cassandra, Hadoop etc. to support online/offline operations.
- Completed certification from Coursera (Stanford) in Machine Learning, thorough understanding of concepts including Supervised/Unsupervised learning, Classification, Anomaly Detection, Recommendation Engines etc.
- Written POCs in Python to analyze the data quickly before applying big data solutions to process at a scale.
- Experience in working with databases, traditional SQL vs NoSQL databases including Oracle, SQL Server, Cassandra, HBase etc.
- Emphasis on usage of design patterns (creational, structural and behavioral) and design principles (TDD, SOLID)
- Experience building Single Page Applications (SPA) using Angular 4, Redux (provides pub/sub between components), Material Controls, Kendo UI, other NPM packages; Typescript and asynchronous operations using Http Client with async/await to consume data exposed through REST APIs.
- Polyglot programmer, experience in developing applications using various languages whether C# & .Net Core and its related frameworks to develop line of business applications using WPF, Composite Application Guidelines (CAG/Prism) with the principles of modularity, separation of concerns, event aggregation, and Model - View-View Model (MVVM) pattern, Service oriented Architecture, data access through Repository Pattern (Entity Framework) etc.
- Usage of Inversion of Controller (IOC) frameworks like Unity to achieve loose coupling through Dependency Injection (DI) with emphasis on TDD.
- Worked on Jenkins Pipelines to build Docker containers and exposure in deploying the same to Kubernetes engine
- Worked on legacy systems including BizTalk server and implemented custom pipelines, adapter customization, business rules, long running transactions, orchestrations with content based routing, sequential and parallel convoy patterns etc.
- Emphasis on code readability, build process and continuous integration
SKILL:
Platforms: Linux/Unix, Windows 7, Windows Server 2008
Languages: C#, Java, Scala, Python
Big Data: Hadoop (Hortonworks, Cloudera, Google Cloud Data Proc), Map/Reduce, Pig, Hive, HBase, Spark, Kafka, Nifi, Cassandra, Intelli J, Maven
Databases: SQL Server 2000/2005 with T-SQL, Oracle with PL/SQL, Cassandra etc.
.Net Ecosystem: .Net Core/Framework, Visual Studio.Net & Resharper, Web API (REST), WCF, WPF, ASP.Net, Unity (IOC), Composite Application Guidelines (CAG/Prism), N-Unit, N-Cover.
Source Control: GIT, Tortoise SVN, TFS
Other: UML with MS Visio, MS Office.
EXPERIENCE:
Confidential, Princeton, NJ
Data Engineer
Responsibilities:
- Server Side data processing:
- Instrumental in various design meetings about architecting data lake, and participated in selection of tools and technologies to use
- Worked on POC to check various cloud offerings including Google Cloud Platform (GCP).
- Compared Self hosted Hadoop with respect to GCPs Data Proc, and explored Big Table (managed HBase) use cases, performance evolution.
- Participated in designing Hive databases, tables, partitioning etc. to store data which ranged to terabytes on a monthly basis
- Have written data pipelines in assisting with batch process of data ingested from various heterogeneous systems including Master Data, Contracts, etc. in different file formats including JSON, text etc.
- Performed streaming data processing of EDI feeds as a result of chassis activity using Kafka, Spark streaming, Cassandra, Hadoop etc.
- Have used Spark RDDs, Spark SQL, Windowing operations etc. to fulfil the business requirements
- Worked on POC to understand Nifi as a general purpose ETL tool, and its pros/cons
- Server Side Services:
- Used Web API to develop REST Services to manipulate data in JSON format for various read and write operations
- Developed Security modules to classify users into various roles with a set of Entities and Permissions, and creating JWT tokens with appropriate claims.
- Used Asynchronous architecture to improve performance of the application and reduce resource utilization including number of background threads as well as reduce processing time
- Used ORM(Entity Framework) to retrieve data from Oracle database and persist the same
- Front End: UI is developed as a SPA (Single Page Application) to assist with EZB and EBM operations
- Used Angular 4 to support various modules with the help of Material Controls, Kendo UI, and other NPM packages
- Extensively used Typescript and asynchronous operations through HTTP Client to fetch data in streams and converted to promises to be awaited on.
- Used Redux to help decouple various UI components in a pub/sub approach and persisted Master Data to Local Storage for persistence and availability across multiple windows.
- Developed Navigation bar to help assist with booking in various stages as in Request/ Review/ Confirmation.
Environment: Windows 10/Linux, Google Cloud Platform (GCP)/MS Azure, Hadoop, Hortonworks HDP 2.6.4, Nifi, Kafka 1.1.0, Spark 2.2, HBase, Hive, AtScale, Power BI, Java, Scala 2.11, Python 2.7, C#, Web API, Angular 5, Typescript 2.7
Confidential, NYC
Full stack Engineer
Responsibilities:
- Using Position, Order, Execution blotters etc. the trader can visualize the trade progress.
- Provides a fully customizable Order Entry window, allowing traders to set different parameters including Destinations (SORT, VWAP, TWAP, Arrival Price etc.), Strategies and different order parameters
- Be able to track market data using Market Watch/ Market Depth controls
- Using Options Watch window can ‘Put and Call’ options available and lift and place orders.
- Ability to check Trade History either through Trade Id, or Symbol or Date range
- Provides various controls to help with trading including Pair Control, Bulk trading etc.
- Server Side Data processing:
- Processed big data using Hadoop cluster consisting of 10s of nodes, analyzing Trade Data generated on daily basis as well as historical.
- Have written Map-Reduce jobs to process data where performance was key
- Used Pig (Pig Latin scripts) to process data on ad-hoc basis
- Enabled report generation by storing and providing data consumed through Hive
- Worked on Entity Model to create various tables/ performance tuning of RDBMS databases.
- Server Side Services:
- Used Asp.Net Web API to develop services to provide data retrieval
- Used Asynchronous architecture to reduce resource utilization.
- Used Entity Framework to retrieve data from external database
- Front End: UI interface was developed for both Intranet Windows Desktop users with WPF as well as external Internet users using Web frameworks
- Developed various WPF Prism modules using MVVM, Event Aggregation etc.
- Developed front end UI application using Angular JS, Asp.Net MVC, Twitter Bootstrap, various Java Script libraries, HTML etc.
- Developed UI using 3rd party controls including Infragistics, Xceed
- Usage Asynchronous programming model to reduce the execution time
- Usage of Reactive framework to consume event-based data that’s pushed onto UI Layer
- Developed the application with SOLID guidelines and used TDD to improve software quality
- Providing timely support by investigating any support issues by analyzing thread dumps, logs generated etc.
Environment: Windows 7/Linux, Cloudera, Hadoop, Map Reduce, Pig, Hive, HBase, Java, Python,, C#, WPF, Web API .
Confidential, NYC
Sr. Developer
Responsibilities:
- Trading module is used to create trade baskets using a ribbon interfaces which helps PMs with workflow involving Uploading transactions, Meeting checklist items, Pre-Trade Analytics, Post-Trade Analytics, and finally Settlement.
- Instruments and hedging modules are used to manage instruments, to perform corporate actions etc.
- And hedging module is designed to help perform rollovers/adjustment to hedge against FX.
- Portfolio Management module is used to manage Portfolios and has various sub modules including Client/Factsheet Reports to asses the performance of the portfolios on a scheduled basis, integration of the portfolio with other external applications (Fixed Income with Barclays Point, Equity with Barra Aegis and Balanced portfolios with in built Asset Allocation module) to help optimize trading.
- Front End: Developed the Beta Engine application using WPF modular architecture
- Architected the user interface using WPF and Prism (Composite Application Guidelines) with each module organized into its own project, worked on root shell which consists of regions to help display more than one view at a time, consisting of multiple regions.
- Developed views using Xaml and used various controls (WPF Datagrid, and 3rd party like Dev Express etc.), and have used data binding, commands, behaviors, dependency/attached properties, styles etc. to build compelling user interfaces.
- Followed MVVM pattern to separate views and view models, used Event Aggregator to interact with external views.
- Used asynchronous operations throughout the application using features including async/await from .Net 4.0 to cleanly separate background process from UI process
- Followed TDD approach and written unit tests to support automated testing of business logic.
- Server Side: Followed n tier application module and developed the services which are hosted on a separate web server farm.
- Developed WCF Services to perform/separate business logic from UI.
- Used Linq2SQL in retrieving the data
- Developed Windows Services to run configurable batch processes.
- Worked on database design, creating entity models and stored procedures etc.
Environment: Windows 7/Server 2003&2008, C#.Net 4.0, WPF, WCF, Linq, Unity, Prism/CAG, NUnit, Linq2SQL, SQL Server 2005, ReSharper etc.
Confidential, Florham Park, NJ
Sr. Developer
Responsibilities:
- ERMS Services: Counterparty Service, Underwriting Service, Claims Service, Accounts Service
- Developed WCF services for various ERMS modules with each module targeting specific business functionality using C#.Net 3.5 and SQL Server 2005.
- Separated the interface from implementation using Contract first approach (service contracts)
- Leveraged WCF transactions and provided transactional support across business and data layers
- Usage of design patterns (abstract factory) and dependency injection framework (Unity) to achieve loose coupling
- Achieved high code coverage with usage of Unity, SOLID principles and mock frameworks.
- SicsNt development using Windows Presentation Framework
- Worked on creating the user interface with XAML, used dependency properties, data binding, styles & templates, etc.
- Followed Composite Application Guidelines (Prism) and implemented Model-View-ViewModel pattern to separate the composition of views and models using ViewModel classes, and Delegate Commands.
- Usage of background threads and dispatcher to keep the user interface active while simultaneously reporting progress and updating the user interface.
- Used event aggregator pattern to communicate between the publisher and subscriber.
- Developed data access layers using LLBLGenPro, an object relational mapper with Linq support which generates POCO classes from tables in database
- Worked on web forms for users to submit policy forms for HVHO policies using Asp.Net
Environment: Windows XP/Server 2003&2008, C#.Net 3.5, WPF, WCF, Linq, Asp.Net, Unity, CAG, NUnit, LLBLGenPro, BizTalk Server 2006 Oracle 9i, SQL Server 2005, ReSharper
Confidential, NYC, NY
Senior Developer
Responsibilities:
- Worked on User Interface for SMF using WPF with the help of Composite Application Guidelines and Unity
- Implemented business layers which involved reducing the processing time by spanning multiple threads, worked on code synchronization, and interthread communication, thread scheduling etc.
- Developed Operations Engine (OE), a service oriented framework created in WCF to enable interoperability between various systems while primary controller providing services like user management, static data, configuration, etc for other applications including trading and market data agents.
- Implemented WCF Per-Call & Singleton Services, Request - Reply & Callback operations, transactions using netTcpBinding
- Achieved high code coverage through Unity, Moq, and NUnit frameworks
- Emphasis on usage of creational, structural and behavioral design patterns
Environment: Windows XP/2003, C#.Net 3.5, WPF, WCF, LINQ, SQL Server 2005, NUnit, Moq.
Confidential, NYC, NY
Senior Developer
Responsibilities:
- Developed the front end application for fund management and worked on Dashboard to implement Workflow Configuration.
- Created web services to support interaction between workflow and fund manager
- Created workflows to support sequential and parallel approval to support fund approval process.
- Created business rules to help the workflow and support changing requirements.
Environment: Windows XP/2003, VS.Net 2005/2008, .Net 2.0/3.5, MSMQ, SQL Server 2005, Team Foundation Server
Confidential, Albany, NY
Enterprise Design Engineer
Responsibilities:
- From the EC 1.6 Design Document, involved in documenting Technical Spec.
- Updated the BizTalk Adapter for both Send and Receive side.
- Worked on configuring BizTalk Orchestrations by creating Send/Receive Ports, Locations, and Binding Files etc.
- Worked on VS Templates to generate BizTalk Projects.
- Coding add-on solutions using .Net which helped in the building the solution.
- From the requirements spec worked on Design Document and Technical Spec.
- Updated the Xml Message Bus to support Generic Messages to support messages other than S95 and B2MML.
- Extensively used C#.Net to develop a Mapping framework to create or modify the Message Type and Message Data without the need to redeploy the Enterprise or Middleware Service.
Environment: Windows XP/2003, C#.Net, SQL Server 2000, Microsoft BizTalk Server 2004, Plant Applications 4.3, Enterprise Connector 1.6/1.7, Xml Message Bus, Tortoise Subversion.
Confidential, Charlotte, NC
Enterprise Design Engineer
Responsibilities:
- As part of Securitizations, extensively used BizTalk Schema Editor for converting the flat files received from various insurance companies and submitting them to central database.
- As part of CIC, used File Adapter to read InfoPath Documents and Orchestrating to create new InfoPath Documents and sending Email Updates.
- As part of Hercules Billing, read CSV documents and submitting them to SQL Server. Good usage of custom XSLT in Orchestrations to do bulk inserts in avoiding repeated roundtrips between BizTalk Server and the Database.
- Being maps are huge and complicated, custom XSLT is used to map inbound and outbound schemas.
- Upgrading the projects from BizTalk 2004 to BizTalk 2006 and creating MSI Installations and Binding Files
- Written custom pipeline components that help in Archiving files, Encryption and Decryption using PGP.
- Participating in peer reviews, unit testing to ensure bug-free solution
Environment: Windows Server 2003, Microsoft BizTalk Server 2004 and 2006, C#.Net, SQL Server 2000, InfoPath, XML/XSLT
Confidential, Phoenix, AZ
Integration Developer
Responsibilities:
- Involved in design phase, been to Amsterdam in finalizing the interfaces between the services offered by BizTalk Server and Oracle Applications. Involved in team discussions and helped in functional specification, transaction recovery and restoring the systems back to normal condition in case of system failure.
- Analyzing the Mapping Documents in creating the Technical Specification
- Creating XML Schema and Property Schema files for inbound & outbound Siebel transactions.
- Mapping inbound and outbound schemas using Maps, Functoids, and Custom Functoids.
- SQL Adapter is used to work with SQL Server while Data Access Application Blocks were used with Oracle 8i.
- Creating custom error handling mechanism to recover and overcome the lost transactions between Siebel and BizTalk Server, BizTalk Server and Oracle.
- Working on installation packages and binding files to deploy and undeploy assemblies.
- Interacting with Siebel and Oracle Team in providing support for production environment
Environment: Windows Server 2003, C#.Net, Siebel CRM 7.5, Oracle 8i, SQL Server 2000Microsoft BizTalk Server with Server Pack 1, Microsoft Data Access Application Blocks, CodeSmith, XML/XSLT
Confidential, Sacramento, CA
Solution Developer
Responsibilities:
- Coordinating with Business Analysts and Technical Team in design and development
- Extending the current workflow application to meet with the project requirements.
- Effectively using FileNET Capture to scan the incoming documents by reading the barcodes and attaching them to the respective work objects by using Jiglue components provided with FileNET (.Net COM Corba Java)
- Designing workflow robots like Document Launcher, Age Timer, Death Completion Process using Windows Services to work with WOBs to perform automated services without user interaction.
- Creating custom libraries for FileNET eProcess (IDM Objects and Jiglue) to avoid late binding.
- Updating the existing workflow application with newly created queues from eProcess like New Case Queue, Ready to Review Queue, Ready to Pay Queue, etc.
- Using web services for interacting with external resources like Oracle Server and Mainframe systems.
- Involving in running through the test cases for BizTalk functoids and long running transactions.
- Effectively using design patterns to simplify the design with elegant and reusable solutions.
Environment: Windows XP, C#, ASP.NET, Microsoft BizTalk Server 2004, Web Services, Windows Services, FileNET eProcess, FileNET Capture, Oracle 9i, SQL Server 2000, VSS, Visio, XML/XSLT.
Confidential, Salem, OR
Software Programmer
Responsibilities:
- Using Windows forms in developing the screens for voter registration and search functionality, fully customizable
- Applying best design practices and tools available to make project development as robust as possible
- Worked with ADO.NET objects like data sets, data tables and data grids.
- Developed 3-Tier application, with business rules developed through stored procedures.
- Worked on data management and manipulation between Relational data and XML.
- Created and used SQL Queries, Stored Procedures in MS SQL server 2000 which helped in faster execution of the client side requests.
Environment: Windows XP, VS.Net, VB.NET, Oracle 9i, WinCVS, Visio.