Important Tools For Hadoop Users

    When it comes to Big Data analysis, Hadoop is at the forefront of things. These tools manage various aspects of big data analysis using Hadoop.

1. Ambari The Apache Ambari project is aimed at making Hadoop management simpler by developing software for provisioning, managing, and monitoring Apache Hadoop clusters. Ambari provides an intuitive, easy-to-use Hadoop management web UI backed by its RESTful APIs. (Official Website) 2. HDFS (Hadoop Distributed File System) The Hadoop Distributed File System (HDFS) is a distributed file system designed to run on commodity hardware. It has many similarities with existing distributed file systems. However, the differences from other distributed file systems are significant. HDFS is highly fault-tolerant and is designed to be deployed on low-cost hardware. HDFS provides high throughput access to application data and is suitable for applications that have large data sets. HDFS relaxes a few POSIX requirements to enable streaming access to file system data. HDFS was originally built as infrastructure for the Apache Nutch web search engine project. HDFS is now an Apache Hadoop subproject. (Official Website) 3. HBase Use Apache HBase when you need random, realtime read/write access to your Big Data. This project's goal is the hosting of very large tables -- billions of rows X millions of columns -- atop clusters of commodity hardware. Apache HBase is an open-source, distributed, versioned, non-relational database modeled after Google's Bigtable: A Distributed Storage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, Apache HBase provides Bigtable-like capabilities on top of Hadoop and HDFS. (Official Website) 4. Hive The Apache Hive ™ data warehouse software facilitates querying and managing large datasets residing in distributed storage. Hive provides a mechanism to project structure onto this data and query the data using a SQL-like language called HiveQL. At the same time this language also allows traditional map/reduce programmers to plug in their custom mappers and reducers when it is inconvenient or inefficient to express this logic in HiveQL. (Official Website) 5. Sqoop Apache Sqoop(TM) is a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases. (Official Website) 6. Pig Apache Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The salient property of Pig programs is that their structure is amenable to substantial parallelization, which in turns enables them to handle very large data sets. (Official Website) 7. ZooKeeper Apache ZooKeeper is an effort to develop and maintain an open-source server which enables highly reliable distributed coordination. (Official Website) 8. NoSQL This is the answer to MySQL and other established database management systems. It includes names like MongoDB etc. 9. Mahout The Apache Mahout project's goal is to build a scalable machine learning library. (Official Website) 10. Lucene/Solr This is an enterprise search platform that comes from the Apache Lucene project. 11. Avro This is a data sterilisation system. 12. Oozie This is a workflow scheduler that helps manage jobs on Apache Hadoop.

 

Eight Things To Avoid In Ubuntu

          

         Ubuntu is clearly one of the most popular Linux distributions right now! While we know what to do with Ubuntu, many don't know the things to avoid in Ubuntu. Have a look..

1. Don’t use installation scripts – Installation scripts like Ultamatix, Ubuntu Tweak, Ubuntu Sources List Generator or Ubuntuzilla are quite risky as they are third party scripts. Listed below are a few of the most dangerous scripts you must avoid -

a) Ultamatix – This is the most harmful of the lot as it can lead to irreparable damage to your system by uploading all sorts of unstable versions of the application. It uses forced permits which include --assume-yes and –force-yes. The only solution to this problems is reinstalling a new version of Ubuntu.

b) Ubuntu Tweak and Ubuntu Sources List Generator – Using these can lead to addition of many PPA's and third party software. There is no clarity regarding where each one comes from and if there is a verification key.

c) Ubuntuzilla – This can lead to bizarre malfunctions in Firefox as the Ubuntuzilla version of Firefox is "original upstream software" that is not totally adapted to and tested for your Ubuntu version.

2. Don’t use cleaning applications – As per sites.google.com, cleaning applications like Computer Janitor or Bleachbit can damage more than you desire. They are not reliable as without you getting any clue about it they will end up removing a lot thereby damaging the system.

3. Avoid using PPAs – The software that is sourced from PPA's (third-party repositories) are not tested in your Ubuntu version. These can cause harm and destabilize your system. Also, it leads you becoming dependent on the sole- owner of the PPA who is not going to be checked.

4. Be careful of using .deb files - The files that have extension .deb are separate installers similar to .exe installers for Windows. They can be downloaded from certain websites by double clicking on them. You will be asked to enter your password and these will be automatically installed in your system. These files are not checked and not verified. They might damage your system as they may have n malware which include spyware etc.

5. Don’t trust Firefox and Chromium/Chrome extensions – You cannot trust these extensions blindly as the more the number of extensions are installed by you the slower Firefox and Chromium/Chrome become. Further, a few add-ons can cause malfunctions with other add-ons or in the browser itself.

6. Avoid mixing desktop environments – Installing Unity/Gnome and complete KDE desktop will convert your PC computer into a mess affecting performance and causing instability and malfunctions. To ensure your operating system’s efficiency install either Ubuntu (Gnome/Unity) or Kubuntu (KDE).

7. Don’t Remove Any Default Application of Ubuntu – The reason being that the default installation is a linked system that depends on shared supporting files accounting for the smooth running of the operating system.

8. Don’t Use Your Production Machine for Experiments – Only experiment on play boxes or dedicated testing machines as it might result in you landing up with an unusable computer. The best thing to do is to only upload LTS versions of Ubuntu on production machines as they are much better and reliable. You can divide your computer chores into three -

- Work horses - Install LTS versions;

- Play boxes - Install newest version of Ubuntu every six months;

- Testing machines – Carry out experiments and bug hunting.

How To Dual-Boot Your Device With Both Android And Ubuntu Touch

        Canonical has announced their dual-boot tool for Android devices but it is a developer preview version             

                      
         You would have noticed that Android-based smartphones and tablets are almost at par with computers. So, just like computers, dual booting these devices is quite possible. In this article, we’ll explain how to dual-boot your device with Android and Ubuntu Touch operating systems. Canonical has announced a dual-boot tool for its OS recently, but being a developer preview, it is not yet completely bug-free. So, use it at your own risk.       


The tool allows you to use both Android and Ubuntu Touch operating systems on your devices. It requires reflashing of the recovery partition of your device. The steps involved in the process are as follows.

But first, you need to meet the following requirements,

- You need an unlocked Android device.
- You need to backup the copy of your current operating system.

Let’s begin.

1. First, start your Ubuntu desktop, add the repository and update it.

sudo add-apt-repository ppa:phablet-team/tools
sudo apt-get update

2. Next you need to install the Developer Preview Tools

sudo apt-get install phablet-tools android-tools-adb android-tools-fastboot

3. If your Android device isn’t unlocked already, then unlock it. This can be done by enabling USB debugging on your Android device and then connect it to Fastboot on your Ubuntu desktop.

sudo fastboot oem unlock

4. Now you need to download the installation script and make it executable.

chmod +x dualboot.sh

5. Now you will be installing the dual booth Ubuntu Installer. This will be done by running the command given below along with your Android device connected to the computer. Here, if you are using a stock ROM, then use the first command. If you’re using a non-stock OS then the second command will come in handy.

- ./dualboot.sh FULL
- ./dualboot.sh UPDATE

A stock ROM is one that has no modifications, like an added SuperUser app etc. A non-stock ROM is like a CyanogenMod edition or something like that.

6. After executing the script, your device may reboot several times. This is normal, so don’t worry about it. When the process is completed, the Ubuntu Installer will be added to the Apps section. In order to install Ubuntu onto it, use the command below,

adb shell “su sh rm -rf /cache/*”
 

7. Open the Ubuntu Installer and choose ‘Trusty’ channel. This will be followed by a number of on-screen instruction that you will need to follow. These will also cover how to provide SuperUser grants. Let the installation continue after this.

8. Once the installation is completed, you will find the ‘Reboot to Ubuntu’ option. Choose this to boot your device with Ubuntu Touch.