PROFESSIONAL EXPERIENCE:

Confidential, New York, NY

Senior Scala and Search Consultant

Responsibilities:

• Enhanced, customized and deployed Elastic Search - based replacement for Google Search Appliance at Confidential of North Carolina.
• The solution is designed as a number of microservices with an Angular 4 SPA UI, completely Dockerized and deployed to AWS ECS as more than a dozen containers.
• It utilizes AWS ECS, Kinesis, Dynamo, S3 and RDS.
• I worked on Scala microservices and on Cloud Formation script customization.

Confidential, Plymouth Meeting, PA

Senior Scala Big Data Architect

Responsibilities:

• Principal architect on a greenfield project to convert legacy analytics solutions to Big Data platform using a proprietary Scala-AKKA-based Spark Job Server Framework, the Job Composer. Some work involved ML Pipelines.
• Tools include Jira, Hue 4, Spark UI, Confluence, Kibana, Elastic Search and Logstash.
• Developed Spark Big Data Analytics using Scala, Spark API, Hive QL and custom udfs.
• Specifically, worked with Confidential Spain Data Science and ML team to enhance and convert an existing R solution to a Spark pipeline for several indications to provide maps of locations most suitable for clinical trials for new medications.
• Also developed several pipelines for the US based on legacy PL/SQL code.
• US solutions required extensive tuning of Spark such as setting various configuration parameters for memory in the driver, executors, numbers of CPUs, broadcast variables and cache settings. Became familiar with tools for analyzing Spark job running performance.

Confidential, Philadelphia

Software Development Manager

Responsibilities:

• I worked on the team that does Big Data analysis of medical data.
• Despite my title, I did 100% development using Scala, Spark, HiveQL and Java 8 on Cloudera Hadoop 2.
• I have developed working knowledge of Hadoop tools like Oozie, Sqoop, Hue, Impala, HBase and some newer ones like Kudu, as well as Hadoop architectural patterns and tradeoffs between Map-Reduce, Tez and Spark. I also learned details and pros/cons of various HDFS file formats like Avro and Parquet.
• I have coded various enhancements and bug fixes to existing Scala Spray Http based RESTful microservices and refactored the codebase to remove code duplication.
• Developed several utility functions, to handle Futures easier and to keep Impala performant under load.
• As part of an effort to move Big Data development to Continuous Delivery model, defined HiveQL best practices, and developed an open source Hive unit-testing framework for HiveQL. It enables Hive developers to write true unit tests of hiveQL code on any platform.
• It also automates Excel-based QA manual certification step necessary before production deployment, by integrating it into a continuous delivery build process.
• This provides automatic regression and QA certification for existing functionality.
• I developed it using Java 8 functional features and not Scala for maintainability by non-Scala developers.

Confidential, Plymouth Meeting, PA

Software Development Consultant

Responsibilities:

• Mentored a team of Hexaware developers from India on Agile development and Scala best practices for a pilot project to explore ways Data Analysis team can use supplier data that will become available in the new IMS data lake.
• Used Scala, Spark, Hadoop 2, Hive, Impala, Elastic Search and Kibana

Confidential, Philadelphia

Software Development Consultant

Responsibilities:

• The team uses Agile process and is part of CI/CD process.
• Main developer of a self-service portal to enable internal clients to provision messaging resources with token-based security similar to AWS STS.
• I was the sole developer from inception and initial two quarters, the first working version was deployed to production three months after the project started in November 2015.
• I developed the project using Domain Driven Design philosophy.
• Designed and evolved the functional domain model and corresponding Play Json marshallers.
• Influenced the team to use more domain driven design approach.
• Defined the architecture and wrote the first release of the Portal based on cats free monads.
• I used the approach described by Debasish Ghosh in the new book ‘Functional and reactive domain modeling’.
• Together with team lead, wrote other portal features using cats co-products, Kleisli and other cats typeclasses.
• Used type-based techniques with shapeless and H-List, to achieve a totally testable system, with all of the business logic being pure and based on co-product of multiple algebras, one for each distinct concern.
• The codebase is DRY, compact and expressive, unlike most Play code I’ve worked with before.
• The system uses the interpreter pattern to achieve full decoupling between the declarative business logic code and the implementation, making the system independent of the storage technology.
• Surprisingly, it is also decoupled from details of the Async abstraction, so Future, Task or anything similar can be used.
• After the first release, evaluated and prototyped some features with Freestyle and FreeK Scala libraries for free monad-based code.
• Learned Elk and developed and configured logstash grok filters to produce logs in a format consumable by existing production monitoring infrastructure.
• Responsible for Couchbase data model and selective use of native data structures such as maps, lists and sets.
• Wrote N1QL queries, designed and optimized indexes, explored Couchbase text search features.

Software Development Consultant

Confidential, Philadelphia, PA

Responsibilities:

• Responsible for architecture and development of Confidential.
• The system is designed to be massively parallel but not real-time and uses Celery to manage concurrent worker processes.
• I developed a prototype in Scala but decided to implement in Python because there are no Scala developers on client’s team to maintain the system.
• The project was developed using Domain Driven Design philosophy.
• The team is agile, uses CI and Dev-Ops to deploy to Amazon AWS.
• Worked with AWS Elastic Beanstalk.
• The system was developed using Doman Driven Design and uses CQRS, Event Sourcing, microservices and View Projection patterns to achieve horizontal scalability.
• All of the business logic is pure and functional. Only the ETL layer uses a couple of classes for validation and reporting.
• The system is storage-agnostic and is configured to work in production with SQL Server, memcached and Postgres (for JSON querying).
• It has been in production for a couple of years now.
• I worked with the front end 5-10% of the time using Django and Angular JS.
• Confidential framework is Nose and BDD is handled by Cucumber.

Vice-President

Confidential, Pennington, NJ

Responsibilities:

• Lead Developer, leading several on and off-shore resources in development of software solutions for clients of Operational Risk group.
• The development process used is SCRUM.
• Developed a workflow façade layer for a new application to manage operational risk.
• The application and the façade layer are written in Python.
• Enhanced the proprietary workflow DSL implemented in Scala with email capability.
• Also worked on UI in Python using Bank’s Quartz infrastructure.
• Diagnosed and fixed performance issues with proprietary Cassandra clone in the new app.
• Wrote an ETL framework in functional Python to validate, transform and clean data that is used by the application since it comes from multiple heterogeneous sources.
• Responsible for investigating newer technologies and developed POC prototypes of the new application in Scala, Play 2 and AKKA on the back end exposing RESTful services and React.js on the client with Couchbase for storage.