AWS

Your Quintessential Guide to AWS Athena

Your Quintessential Guide to AWS Athena

Serverless has become a new trend today and is here to stay for sure! Now when you think of wireless internet, you know that it still has some wires but you don’t need to worry about them as you don’t have to maintain them. Similarly, serverless has servers but you don’t have to keep worrying about handling or maintaining them. All you need to do is focus on your code and you’re good to go.

It has some more benefits, such as:

  • Zero administration: You can deploy code without provisioning anything beforehand, or managing anything later. There is no concept of a fleet, an instance, or even an operating system.

  • Auto-scaling: It let’s your service providers manage the scaling challenges. You don’t need to fire alerts or write scripts to scale up and down. It handles quick bursts of traffic and weekend lulls the same way.

Managing Secrets Using AWS Systems Manager Parameter Store and IAM Roles

Managing Secrets Using AWS Systems Manager Parameter Store and IAM Roles

Amazon Web Services(AWS) has an extremely wide variety of services which cover almost all our infrastructure requirements. Among the given services, there is AWS Systems Manager which is a collection of services to manage AWS instances, hybrid environment, resources, and virtual machines by providing a common UI interface for all of them.

Services are divided into categories such as Resource Groups, Insights, Actions and Shared Resource. Among Shared Resources one is Parameter Store, which is our topic of discussion today.

Real Time Analytics for IoT Data using Mosquitto, AWS Kinesis and InfluxDB

Real Time Analytics for IoT Data using Mosquitto, AWS Kinesis and InfluxDB

Internet of things (IoT) is maturing rapidly and it is finding application across various industries. Every common device that we use is turning into the category of smart devices. Smart devices are basically IoT devices. These devices captures various parameters in and around their environment leading to generation of a huge amount of data. This data needs to be collected, processed, stored and analyzed in order to get actionable insights from them. To do so, we need to build data pipeline.

In this blog we will be building a similar pipeline using Mosquitto, Kinesis, InfluxDB and Grafana. We will discuss all these individual components of the pipeline and the steps to build it.

Jenkins X - A Cloud-native Approach to CI/CD

Jenkins X - A Cloud-native Approach to CI/CD

Jenkins X is a project which rethinks how developers should interact with CI/CD in the cloud with a focus on making development teams productive through automation, tooling and DevOps best practices.

In this blog, we explore Jenkins X, understand how it differs from Jenkins and how to go about building and deploying our first application using it.

Deploy Serverless, Event-driven Python Applications Using Zappa

Deploy Serverless, Event-driven Python Applications Using Zappa

Zappa makes it super easy to build and deploy server-less, event-driven Python applications (including, but not limited to, WSGI web apps) on AWS Lambda + API Gateway. Think of it as "serverless" web hosting for your Python apps. That means infinite scalingzero downtimezero maintenance - and at a fraction of the cost of your current deployments!

This blog acts a simple tutorial to deploy a sample Django/Python application using Zappa.

Lessons Learnt While Building an ETL Pipeline for MongoDB & Amazon Redshift Using Apache Airflow

Lessons Learnt While Building an ETL Pipeline for MongoDB & Amazon Redshift Using Apache Airflow

Recently, the author was involved in building a custom ETL(Extract-Transform-Load) pipeline using Apache Airflow which included extracting data from MongoDB collections and putting it into Amazon Redshift tables. 

Each ETL pipeline comes with a specific business requirement around processing data which is hard to be achieved using off-the-shelf ETL solutions. This is why a majority of ETL solutions are built manually, from scratch. In this blog, I am going to talk about my learnings around building an optimized, efficient, near real-time and fault tolerant custom ETL solution using Apache Airflow which involved moving data from MongoDB to Redshift.

Web Scraping: Introduction, Best Practices & Caveats

Web Scraping: Introduction, Best Practices & Caveats

Web Scraping is the process of data extraction from various websites present over the internet. Web Scraping has a wide variety of use cases. Marketing & Sales Intelligence companies use web scraping to fetch customer related information, Real Estate Tech companies use web scraping to fetch real estate listings, Price Comparison Portals use web scraping to fetch product and price information from various e-commerce sites. This blog is meant to be a primer on building highly scale-able scrappers. The blog will cover different ways to scrape, how to scrape at scale and guidelines while writing scrappers.

A Quick Guide to Building a Serverless Chatbot With Amazon Lex

A Quick Guide to Building a Serverless Chatbot With Amazon Lex

Amazon Lex is a AWS service for building conversational interfaces into any application using voice and text. Amazon Lex provides the advanced deep learning functionalities of automatic speech recognition (ASR) for converting speech to text, and natural language understanding (NLU) to recognize the intent of the text, to enable you to build applications with highly engaging user experiences and lifelike conversational interactions. 

This blog is a detailed step-by-step tutorial for developing smart chatbots with serverless functions (Amazon Lambda).

Serverless Computing: Predictions for 2017

Serverless is the emerging trend in software architecture. 2016 was a very exciting year for serverless and adoption will continue to explode in 2017. This post covers the interesting developments in serverless space in 2016 and our thoughts on how this space will evolve in 2017.