Gradient Boosting Decision Tree

In the previous article, we've talked about AdaBoost which combines output of weak learners into a weighted sum that represents the final output of the boosted classifier. If you know little about AdaBoost or additive model, we highly recommend you read the article first. Gradient boosting is a machine learning technique for regression and classification problems, which produces a prediction model in the form of an ensemble of weak prediction models, typically decision trees. It builds the model in a stage-wise fashion like other boosting methods do, and it generalizes them by allowing optimization of an arbitrary differentiable loss function. Boosting Tree Boosting tree is based on additive model which can be repsentated as following: \(\begin{align}f_M(x) = \sum_{m=1}^{M} T(x; \theta_m)\end{align}\) where \(T(x; \theta_m)\)

Continue Reading

AdaBoost

AdaBoost, short for "Adaptive Boosting", is a machine learning meta-algorithm formulated by Yoav Freund and Robert Schapire who won the Gödel Prize in 2003 for their work. The output of the other learning algorithms (weak learners) is combined into a weighted sum that represents the final output of the boosted classifier. AdaBoost is adaptive in the sense that subsequent weak learners are tweaked in favor of those instances misclassified by previous classifiers. AdaBoost is sensitive to noisy data and outliers and is quite robust to overfitting. Bagging vs. Boosting A too complex model (unpruned decision trees) have high variance but low bias whereas a too simple model (Weak learners like decision stumps) have high bias but low variance. To minimize…

Continue Reading

Random Forest

Random forests are an ensemble learning method for classification, regression and other tasks, that operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees. Random decision forests correct for decision trees' habit of overfitting to their training set. Random Forest builds many trees using a subset of the available input variables and their values, it inherently contains some underlying decision trees that omit the noise generating variable/feature(s). In the end, when it is time to generate a prediction a vote among all the underlying trees takes place and the majority prediction value wins. Ensembles are a divide-and-conquer approach used…

Continue Reading

Setup Raspbian on Raspberry Pi

This resource explains how to install Raspbian operating system and setup a Raspberry Pi. You will need another computer with an SD (or MicroSD) card reader to install the image. Install Raspbian Download the Image Raspbian is the Raspberry Foundation’s official supported operating system and can be download from Raspberry Pi website Downloads page. After downloading the .zip file, unzip it to get the image file (.img) for writing to your SD card. And you can write this image to SD card with the help of Win32 Disk Imager in Windows. Setup for HDMI Output Once the write is complete, edit config.txt in boot partition as following for HDMI output. Once you’ve steps above, place the SD card into your Raspberry…

Continue Reading

Internet Protocol v6

Internet Protocol version 6 (IPv6) is the most recent version of the Internet Protocol (IP) which was developed by the Internet Engineering Task Force (IETF) to deal with the long-anticipated problem of IPv4 address exhaustion. IPv6 is intended to replace IPv4. IPv6 uses a 128-bit address, theoretically allowing 2128, or approximately 3.4×1038 addresses. The actual number is slightly smaller, as multiple ranges are reserved for special use or completely excluded from use. The total number of possible IPv6 address is more than 7.9×1028 times as many as IPv4, which uses 32-bit addresses and provides approximately 4.3 billion addresses. The two protocols are not designed to be interoperable, complicating the transition to IPv6. However, several IPv6 transition mechanisms have been devised to…

Continue Reading

Setup IPv6 NAT on OpenWRT Router

Introduction This tutorial is a HowTo for setting up IPv6 NAT on OpenWRT router.   The information of the environment is listed below. Network Environment: China Education and Research Network Center with dual stack IPv6 Network Device: NETGEAR R6100 (128M RAM) Firmware Version: OpenWrt Chaos Calmer 15.05 Install Dependencies for OpenWRT Login to OpenWRT router via SSH and run following commands: Setup IPv6 for Local-area Network Edit /etc/config/network, and add following lines in config interface 'lan': Remove following lines if exists: Edit /etc/config/dhcp, and change the content in  config dhcp 'lan' as follows: Setup Gateway for IPv6 Network Add following lines to /etc/firewall.user for forwarding IPv6 traffic: Setup up IPv6 gateway by creating file in /etc/hotplug.d/iface/90-ipv6 and adding following lines to the file: Then, add execute permission…

Continue Reading
Contact Us
  • Room 311, Zonghe Building, Harbin Institute of Technology
  • cshzxie [at] gmail.com