Hadoop Engineer / Bigdata Specialist/architect Resume
2.00/5 (Submit Your Rating)
New York, NY
SUMMARY:
- An IT Specialist/Architect with strong analytical and technical background in Apache Hadoop, WebSphere products / WebSphere Portal, Confidential pSeries, Linux, AIX, AIX HACMP, LPAR, Solaris, Sun Cluster, Veritas Cluster, Veritas Volume Manager, Veritas File System, Systems Integration, Unix/C Programming Language, Shell scripting.
- Currently leveraging strong cross - competency knowledge to deliver Big Data and analytics solutions.
- Current position as an IT Architect/Specialist in Confidential offers intimate knowledge and expertise of architectures and delivery processes.
- Proven track record of strong communication, leadership skills. Excellent analytical and problem-solving skills. Recognized high performance contributor.
- Over 15 years of IT industry experience encompassing a wide range of skill set, roles and industry verticals.
- Experience in leading and managing teams and excellent team spirit qualities. Proficient in analyzing and translating business requirements to technical requirements and architecture.
- Extensive experience with user requirement analysis, solution, configuration, design, system testing, performance tuning, software migration, implementation and maintenance of Apache Hadoop cluster, WebSphere products and Confidential hardware.
- Industry experience includes finance, production, insurance, distribution/retail and health.
- Extensive Hands on experience in installation, configuration, troubleshooting and managing Hadoop Clusters
- Deep understanding of Hadoop architecture, HDFS, HBase, Hive, Spark, Oozie, Zookeeper, Yarn, BIGSQL and other components in the Apache Hadoop eco-system
- Experience with Hadoop cluster capacity planning, performance tuning, cluster monitoring and troubleshooting
- Extensively used Ambari console to manage Hortonworks Hadoop cluster
TECHNICAL SKILLS:
- eServer: -pSeries, xSeries, LPAR
- Sun - SunFire 6800, 3800, V880, E450, 480R, 280R.
- Confidential WebSphere Application Server, Confidential WebSphere Portal, Confidential HTTP Server, Confidential Tivoli Directory Server, Cisco and Alteon Load Balancer configuration.
- AIX HACMP, Logical partitioning (LPAR), HMC, Sun Cluster, Veritas Cluster, Veritas Volume Manager, Veritas File System, System Integration, NFS (Network File System), NIS, System Performance Tuning, System HW/SW Components
- Languages: C, C++, Visual Basic, Pascal, Fortran, Java Script.
- Editors/Scripts: Vi and Emacs editor; Advanced Shell scripting, perl and python.
- Big Data Components: HDFS, Hive, Pig, Sqoop, Oozie, Zookeepr, Yarn, HBase, R & BigR,
- Databases and Tools: Oracle, DB2, MySQL, BIGSQL, SQL Server, MongoDB.
- Expert knowledge of large storage XP/EMC arrays/SAN concepts.
PROFESSIONAL EXPERIENCE:
Confidential
Hadoop Engineer / BigData Specialist/Architect
Responsibilities:
- Assist Sales team to provide technical architecture related to the Confidential Big Data and Analytics System specific to the customer requirements
- Configure, build, test, validate and delivery of Confidential BigInsights and Hadoop clusters to the customer
- Work with Confidential partners to migrate other related products such as Confidential Spectrum Scale along with BigInsights 4.2 migration project.
- Work with various development teams across different continents such as US, India, and Canada for BigData projects. Also performed knowledge transfer to Hadoop administrators and work as a technical lead across different continents.
- Develop/write PERL and KORN shell scripts for metadata migration tasks.
- Creates schemas and tables for clients as requested. Run DB2 explain and db2 support to trouble shoot customer SQL related queries.
- Trouble shoot BIGSQL related issues.
- Work with client database administration team to develop processes and procedures for disaster recovery which included DB2 data replication.
- Setup ranger KMS and create encryption zones within HDFS as well assign specific access for proper data security.
- Migrate BigData production clusters IOPs and value packs packages from version 3 to version 4.2.5 for various clients over the weekend with in 24 hours window.
- Work with product team to open PMRs for defects and work on the resolution, testing and implementation.
- Apply special builds for BIGSQL which includes testing and validations and coordination with clients.
- Install and configure various version of Hortonworks, BigInSights and Cloudera clusters via Ambari and Cloudera UI.
- Work on installing and configuring Hadoop clusters from begging to end which includes setting up the Linux environments, Password less SSH and all pre-installation work.
- Enabling LDAP and SSL for various components of Hadoop cluster.
- Proactively monitor cluster health and performance tuning activities
- Managing and reviewing Hadoop log files and debugging failed jobs
- Commissioning & decommissioning of data nodes and name node recovery
- Worked with clients to apply Hadoop updates, patches, version upgrades as required
- Help clients loading external data to Hadoop environments using tools like Sqoop, Flume and Kafka
- Responsible for support of Hadoop Production environment which includes Hive, YARN, Spark, BIGSQL, Kafka, SOLR, Oozie, Sentry, Encryption, Hbase, etc.
- Configuring yarn and BIGSQL memory settings and yarn queues as required.
- Automate jobs for moving data from FTP server to load data into Hive tables, using Oozie workflows
- Assist business Intelligence team by partitioning and querying the data in Hive
- Work with developers and end users to resolve business critical problems and keep management up to date on outstanding Big Data / Hadoop cluster related issues
Confidential
IT Architect/ IT Specialist
Responsibilities:
- Work as a focal point for Unix team, application developers, Data Base team and customers as well.
- Set up meetings, schedule project date and understand customers project requirements. Write procedures and documentations for various projects for administrators to follow.
- Perform various Application load testing to help sizing and planning hardware and software requirements for customer’s potential new environments.
- Also, provide advises and help develop the architecture for new customers/environments and get involve in pre and post sales activities.
- Worked on Confidential Websphere Portal Server versions 5, 6 & 6.1 development, design & administration and as well as Confidential Websphere application server version 3.5 4, 5, 6 & 6.1
- Developed an overall upgrade procedure for WebSphere Application Server and WebSphere Portal and Confidential Directory server 6.1 and successfully migrated 10 different portal cluster environments with many nodes.
- Worked as a project lead and worked with different teams such as DB2, Application developers and Unix Admin and customers.
- Work closely with Account team and Project executives on technical business issues and deliver technical advice and counsel, define solution requirements, Specify highly-heterogeneous hardware and software components as well as lead and deliver new projects.
- Travel to different Confidential locations or even customer site to work with customers, Developers, other Confidential team and help design application layout, Design high-availability (HACMP), load-balanced e-business solutions for commercial customers.
- Apply strong understanding of business requirements and offerings to deliver best-of-breed web hosting solutions.
- Strong communication, consulting, and client relationship skills.
- Demonstrate highest commitment to customer satisfaction and business requirements. Provide technical feedback to management to facilitate decision-making.
- Add value to standards and processes by suggesting improvements which eliminate rework and reduce transition time.
- Provide guidance to cross-matrix teams regarding delivery architecture best-practices.
- Successfully provide Delivery Architect/support services for: Sprint, AIG Insurance, Sony, Nationwide, Lowes, Starwood Hotels, Nisource Energy, Duke Energy and many more.
- Installed /configured/designed WebSphere Application Server, WebSphere Network Deployment server and WebSphere Portal Server, LDAP directory Server, portal clusters and WebSphere clusters on all type of unix platforms.
- Inform account team about security advisories and their impact and advise possible fixes for AIX and WebSphere and other middleware products such as IHS, Apache, WebSphere Portal, Tivoli Directory servers.
- Deploy application for WebSphere Application server, WebSphere cluster and WebSphere Portal cluster.
- Implemented /wrote scripts to automate various tasks such as security health check scripts for WebSphere, IHS and Apache, application code deployment, User ID management related tools/scripts and etc.
- As a technical lead, engage any critical issue, customer outage and critcit, use the best problem determination skills, troubleshoot problems and help finding the root cause of a issue and provide the best solutions.
- Recipient of multiple High Performance Services awards.
- Received WebSphere SME appreciation from Confidential Taag (Technology at a Glance) team for assisting as a Subject Matter Expert.
Confidential
UNIX System Administrator / IT Specialist
Responsibilities:
- Provided server support for Confidential pSeries & xSeries and SUN hardware as well as AIX, Solaris and Linux operation systems for about more than thirty different customers such as City Group, AIG, Sony e-solution, United Health Care, ADVO, Athena Insurance, Confidential, ACE Insurance and many more.
- Prepared presentations on various topics of Solaris, Veritas Volume Manager and AIX and presented to the account team and project executives to define solution requirements.
- Applied strong understanding of business requirements and offerings to deliver best-of-breed web hosting solutions for Confidential hardware and software solutions.
- Developed Backup/Recovery strategy for most of the customers.
- Experienced in giving backup plan and Recovered databases from Application Migration failures and involved in many Disaster Recovery Exercises.
- Created of Build server for Sun, AIX and Linux to provide different type of build images for different customers.
- Provided evaluation of security Vulnerability Advisories and implement the fix/patches as appropriate.
- Defined solution for security auditing, security hardening, policy-setting, and source-based software installation (OpenSSL, OpenSSH, TSM and health checks for WebSphere & UNIX (AIX, Sun).)
- Built new servers and cluster environments for AIX ( 5.1 & 5.2 ), Sun (2.6 to 9.0) and Red hat Linux (AS and ES).
- Installed & Configured p590 with SSAs, SAN based Tivoli Storage Management Backup solutions using Confidential 3584 Ultrium Tape Library.
- Installed and configured LPAR and dynamically created portioned on pSeries machines such as p690 and etc.
- Designed, installed, configured and support HACMP, Veritas Cluster and Sun Cluster for various customers on AIX and Solaris platforms..
- As a technical lead, helped troubleshoot hardware and software errors on various platforms, HACMP, Veritas volume manager/file system questions and fix issues.
- Designed and implemented a central tool called Cronus to automate jobs which saved manual labor and improved efficiency and turn around time.
- Wrote many shell and perl scripts to automate jobs.
- Wrote scripts to configure HACMP and monitor Applications to provide the best quality services and deliver Confidential best-of-breed n-tier commercial web hosting services.
- Contributed to the creation of cross-training education plan for the USF Generalist Pilot and helped building test lab at the lowest possible cost.
- Tested, created document, update as needed and transition the AIX Alternate Disk Migration process from AIX 4.3 to AIX 5.1 or 5.2 and upgraded about 70% of the infrastructure and a good amount of customer servers as well.
- Wrote a scripts that made the AIX migration process simpler and time efficient.
- Attended techinical meetings with my team and other teams and communicated and helped resolving issues.
- Insatlled and configured CSS Cisco and Alteon Load Balancers.
- Support and troublesoot any middlewares (Apache, Iplanet, WebSphere, IIS and HIS) or Load Balancers issue.
- Supported new middleware transition such as installed Websphere products, configured Network deployment server, tuned/configured application servers, configure JDBC and etc.
- Coordinated and lead and provide technical support for user ID administration team.
Confidential, New York, NY
IT Support
Responsibilities:
- Intalled solaris software version 7 and 8 and configured servers.
- Installed Confidential AIX version 4.3 and version 5.1 and configured servers.
- Intalled software packages and patches in unix servers.
- Jointly involve in maintaining Novell / NT-based LAN / WAN of 100 computers and 300 users.
- Installed down-uploaded software and backed up and restored file systems.
- Developed and document technical procedures used by other team members
- Upgraded database in NT Servers · Troubleshoot software and hardware · Maintain mail server
- Configured and installed software in the laptops for users.
- Upgraded database in the NT Servers
- Downloaded data from Main Frame to NT Server using FTP
- Created batch files to run the applications.
- Produced Operational Analysis Report (OAR) nightly
Confidential, New York
Lab Assistant
Responsibilities:
- Installed, upgraded software and hardware in NT and Unix Lab.
- Supported users, installed new systems and configures as needed.
Confidential
Tutor
Responsibilities:
- Instructed students in math and computer science via individual tutoring.