Lei Feng 的 Blog

This post describes the process to install Hortontworks HDP 3.1.0 on a cluster of three VMWare virtual machines. The process includes four major steps: 1) set up the cluster environemnt; 2) set up a local repository for both Ambari and HDP stacks; 3) Install Ambari server and agent; 4) install, configure and deploy the cluster. This installation process might work for other versions too. Please check the product versions through Hortonworks support matrix: https://supportmatrix.hortonworks.com/ Read More...

With the Apache Spark installed through the steps described in last post, this post will introduce you the steps to set up a Scala development environment for Spark and build a WordCount application through Maven and SBT. Althrough Spark can be programmed with either Java, Scala, or Python, this post will focus on Scala. There are a couple of reasons: 1) Spark itself is written in Scala; 2) Scala’s functional programming model is a good fit for distributed processing, thus has less code and boilerplate stuff than Java; 3) Scala compiles to Java bytecode, which gives faster performance than Python. Read More...

As a data scientist, I often need to run Python scripts on Linux servers. The vast majority of CentOS/RHEL-based Linux servers use Python 2.6.6, while most of my Python applications are written in Python 3. In additoin, on most of the servers, I don’t have the sudo priviledge, so it needs a little trick to install Python 3 on those servers. This post is a step-by-step instructions on installing Python 3.6 through both Anaconda and Python Gzipped source tarball file on a CentOS/RHEL-based Linux server. Read More...

Flask is a micro framework for Python and based on Werkzeug and Jinja2 template engine for developing web applications. With the Azure VPS that has been configured in last post, in this post I will go through steps to deploy a flask application on the VPS. Read More...

Apache HTTP Server is the most used web server software in the world and public cloud services, such as Amazon Web Services and Microsoft Azure, make setting up a Virtual Private Server(VPS) hosting web applications a lot easier than traditional ways that use physical servers. Unlinke many other tutorials that focus on traditional web hosts and configurations for running PHP and/or .NET applications, this post presents steps of installing and configuring Apache HTTP Web Server on an Azure Ubuntu Virtual Machine to host Flask Web applications. Read More...