Category Archives: Tech

Yik Yak Auto Downvote Script

For research purposes… ¯\_(ツ)_/¯

window.scrollBy(0, 10000);

function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
async function downvote() {
var L = document.getElementsByClassName(“downvote”);
var i = 0;
for (i=0; i<L.length; i++) {
console.log(“Downvoting…”);
setTimeout(L[i].click());
await sleep(2000);
window.scrollBy(0, 10000);
}
downvote();
}
downvote();

Installing Hadoop 2.6.X on Raspberry Pi B Raspbian Jessie

Configure Java Environment

With the image Raspbian Jessie image, Java comes pre-installed. Verify by typing:

java -version

java version "1.8.0"
Java(TM) SE Runtime Environment (build 1.8.0-b132)
Java HotSpot(TM) Client VM (build 25.0-b70, mixed mode)

Prepare Hadoop User Account and Group

sudo addgroup hadoop
sudo adduser --ingroup hadoop hduser
sudo adduser hduser sudo

Configure SSH

Create SSH RSA pair keys with blank password in order for hadoop nodes to be able to talk with each other without prompting for password.

su hduser
mkdir ~/.ssh
ssh-keygen -t rsa -P ""
cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys

Verify that hduser can login to SSH

su hduser
ssh localhost

Go back to previous shell (pi/root).

Install Hadoop

Download and install

cd ~/
wget http://apache.cs.utah.edu/hadoop/common/hadoop-2.6.4/hadoop-2.6.4.tar.gz
sudo mkdir /opt
sudo tar -xvzf hadoop-2.6.4.tar.gz -C /opt/
cd /opt
sudo mv hadoop-2.6.4 hadoop
sudo chown -R hduser:hadoop hadoop

Configure Environment Variables

This configuration assumes that you are using the pre-installed version of Java in Raspbian Jessie.

Add hadoop to environment variables by adding the following lines to the end of /etc/bash.bashrc:

export JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")
export HADOOP_INSTALL=/opt/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin

Alternative you can add the configuration above to ~/.bashrc in the home directory of hduser.

Exit and reopen hduser shell to verify hadoop executable is accessible outside /opt/hadoop/bin folder:

exit
su hduser
hadoop version

hduser@node1 /home/hduser $ hadoop version
Hadoop 2.6.4
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152
Compiled by mattf on Mon Jul 22 15:23:09 PDT 2013
From source with checksum 6923c86528809c4e7e6f493b6b413a9a
This command was run using /opt/hadoop/hadoop-core-2.6.4.jar

Configure Hadoop environment variables

As root/sudo edit /opt/hadoop/conf/hadoop-env.sh, uncomment and change the following lines:

# The java implementation to use. Required.
export JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")

# The maximum amount of heap to use, in MB. Default is 1000.
export HADOOP_HEAPSIZE=250

# Command specific options appended to HADOOP_OPTS when specified
export HADOOP_DATANODE_OPTS="-Dcom.sun.management.jmxremote $HADOOP_DATANODE_OPTSi -client"

Also, we need to edit “yarn-env.sh”. Uncomment

#export YARN_NODEMANAGER_OPTS

and write:
export YARN_NODEMANAGER_OPTS=”-client”

Note 1: If you forget to add the -client option to HADOOP_DATANODE_OPTS and/or YARN_NODEMANAGER_OPTS you will get the following error messge in hadoop-hduser-datanode-node1.out:

Error occurred during initialization of VM
Server VM is only supported on ARMv7+ VFP

Note 2: If you run SSH on a different port than 22 then you need to change the following parameter:

# Extra ssh options. Empty by default.
# export HADOOP_SSH_OPTS="-o ConnectTimeout=1 -o SendEnv=HADOOP_CONF_DIR"
export HADOOP_SSH_OPTS="-p <YOUR_PORT>"

Or you will get the error:

connect to host localhost port 22: Address family not supported by protocol

Configure Hadoop

In /opt/hadoop/conf edit the following configuration files:

core-site.xml

<configuration>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/hdfs/tmp</value>
  </property>
  <property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:54310</value>
  </property>
</configuration>

mapred-site.xml

<configuration>
  <property>
    <name>mapred.job.tracker</name>
    <value>localhost:54311</value>
  </property>
</configuration>

hdfs-site.xml

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
</configuration>

Create HDFS file system

sudo mkdir -p /hdfs/tmp
sudo chown hduser:hadoop /hdfs/tmp
sudo chmod 750 /hdfs/tmp
hadoop namenode -format

Start services

Login as hduser. Run:

/opt/hadoop/sbin/start-dfs.sh
/opt/hadoop/sbin/start-yarn.sh

Run the jps command to checkl that all services started as supposed to:

jps

16640 JobTracker
16832 Jps
16307 NameNode
16550 SecondaryNameNode
16761 TaskTracker
16426 DataNode

If you cannot see all of the processes above review the log files in /opt/hadoop/logs to find the source of the problem.

Google Photos Compression

In case you had any doubts about the definition, Google Photos compresses photos if you select the ‘High Quality’ option, even for photos under 16 MP. What they mean is that photos above 16 MP will be visibly more blurry after compression than those under 16 MP. For photos under 16 MP, you will not be likely to notice the difference unless you zoom in using your computer or blow up the image to poster size.

In sum, if you do not want your photos to be touched at all, choose the Unlimited option. Google Photos do compress photos below 16 megapixels under the ‘High Quality’ mode.

How to get uTorrent on Google Cloud Engine

Here are the steps I used to torrent on Google App/Cloud Engine with Debian 7 as its OS.

wget http://download-new.utorrent.com/endpoint/utserver/os/linux-x64-debian-7-0/track/beta/ --content-disposition
tar xf utserver.tar.gz
mv utorrent-server-* ~/utorrent
cd ~/utorrent
wget http://droidzone.in/utstart
chmod +x utstart
./utstart install
./utstart start &

And you’re done!

Next, you need to open up port 8080, under the Networks tab in your console. You will be able to see your uTorrent Remote GUI at http://yourhostname:8080/gui/

Of course you want to download more than 10GB of stuff on your image. You will need to attach a secondary disk to your main VM. To this end,

mkdir /new-disk
mount -t ext2 /dev/sdb /new-disk

to mount the new disk (usually sdb)  to the new-disk endpoint.

There! Now use up your $1000 or $2000 Google credit. 😀

Singapore’s Version of Iron Dome

I just had a random brainwave today to build portable air defence systems atop HDB flats around Singapore. Perhaps this can form the basis for a future research project at CMU! 😀

iron-dome

An interesting article from Haaretz.

A Paris-based online magazine covering intelligence and security issues this week called Singapore one of the most important customers of Israel’s defense industry, laying bare the active, though secret, relationship between Israel and Singapore that began more than 40 years ago – a statement that comes after years in which Israel censored all local articles on the subject.

Intelligence Online, which is published in English on a bimonthly basis, states that the Southeast Asian island state helped finance the Iron Dome system designed by Israel’s Rafael Advanced Defense Systems to intercept short-range missiles and rockets, in exchange for which it is supposed to receive several Iron Dome systems to deploy on its territory.

Even more interesting is the possibility the article raises that Iron Dome was designed first and foremost for the benefit of Singapore – not for the protection of Sderot and the southern communities in Israel that suffered from Qassam rocket attacks and mortar fire for seven years and are still suffering (though Iron Dome is not capable of intercepting mortar shells).

Israeli media have previously hinted at this, but the Defense Ministry has vehemently denied it.

The suspicions were bolstered by the fact that after the system was developed and one battery had been deployed as an Israel Defense Forces base, it turned out that the Defense Ministry had no budget for additional batteries. In that case, why was there a need to develop a system for which there is no budget and which the IDF does not intend to deploy?

According to Intelligence Online, which focuses on arms transactions between countries and corporations and on appointments of intelligence personnel and their clandestine activity, the Iron Dome transaction is the latest between Israel and Singapore.

The Web site, whose articles are available only to paid subscribers, has thousands of readers, including Israelis.

Iron Dome, which its developers said was tested successfully a few months ago, as Israeli media have previously reported, cost roughly $250 million to develop.

One battery, whose production cost is about $50 million, has already been deployed at a base in the south of the country, but so far has not been readied for operational purposes and has not yet been activated.

The anti-aircraft division of the Israel Air Force, which is responsible for operating Iron Dome, is training teams at a base in the north.

They will be operating the system in Israel, with the aim of intercepting Qassam and Katyusha rockets up to a distance of 40 kilometers.

Vulcan-Phalanx: cheaper and more accessible

Intelligence Online also repeats an argument published in recent years in Israel to the effect that if the Defense Ministry had really wanted to protect the residents of the south quickly and cheaply, it could have acquired a cheaper and more accessible defense system than Iron Dome: the batteries of the Vulcan-Phalanx cannon system manufactured by Raytheon.

The land-based version of the batteries, called Centurion, are deployed in Afghanistan and Iraq, where they are used to protect American and NATO forces.

Although Defense Minister Ehud Barak has told Haaretz several times that Israel will acquire the Vulcan-Phalanx system, that has yet to happen.

In other words, the Defense Ministry may have given Rafael a development budget as a way of positioning the project as an Israeli military system that is ostensibly being used by the IDF but is really aimed at improving Israel’s chances of selling it to Singapore and other countries.

Small country, hostile population

The cooperation between Israel and Singapore rests on the two small countries’ shared sense of being under threat, since both are surrounded by a hostile Muslim population and want advanced weapons systems to maintain a qualitative advantage over their neighbors.

The Intelligence Online article argues that the fight against fundamentalist Islamic terror over the past decade has increased the cooperation between the two countries, as well as their sense of a shared destiny. In recent years, Singapore has confronted threats by Jemaah Islamiyah, a terrorist group that operates in Southeast Asia.

The island state, a neighbor of Indonesia and Malaysia, has arrested dozens of the group’s operatives and exposed plans to attack the Israeli, American and Australian embassies, along with ships from those countries. One of Singapore’s main sources of income is the Port of Singapore, which claims to be the busiest port in the world.

According to the article, immediately after Singapore declared its independence in 1965 it asked Israel to help it establish an army. IDF officers including Rehavam Ze’evi (who became a right-wing cabinet member assassinated in 2001) and Benjamin Ben-Eliezer (now the industry, trade and labor minister) were sent to Singapore to head large delegations of military advisers, and helped build the army on the model of the IDF. Israeli military representatives have been active since then at the Israeli Embassy in Singapore, which was opened in 1969.

One of the issues the IDF representatives deal with is promoting large arms deals. Transactions mentioned in the article include Singapore’s purchase of Barak surface-to-air missiles manufactured by Israel Aerospace Industries and Israel’s upgrading of fighter planes belonging to Singapore’s air force.

In addition, Rafael supplied drones for naval missions and Israel’s Elbit Systems supplied its Hermes drone.

Intelligence Online also says there is naval cooperation between the two countries, and notes that the commander of Israel’s navy, Admiral Eli Marom, had previously represented Israel in Singapore.

 

As well as from UPI

Singapore is reported to be acquiring Israel’s new Iron Dome anti-missile air-defense system under a once-secret military cooperation pact with the Jewish state that dates to the 1960s.

Indeed, according to the Paris Intelligence Online Web site, Singapore helped finance the development of the system by Rafael Advanced Defense Systems.

It says that Iron Dome was always intended for the island, a key Asian shipping hub and one of the richest per capita states in the world, because of its strategic location as a trading center.

The Israeli air force, which runs the Jewish state’s air defense network, completed test-firings of the system in January and the first battery is currently becoming operational.

Israeli media reports say the missiles, designed to shoot down hostile missiles with ranges of up to 25 miles, are being deployed along Israel’s northern border with Lebanon.

There are growing fears of a new Middle Eastern war and Hezbollah, the Iranian-backed Shiite movement in Lebanon, has an arsenal of up to 45,000 rockets and missiles, Israeli Defense Minister Ehud Barak has claimed.

Israel’s defense links with largely Chinese Singapore go back to 1965, shortly after the island city-state, a former British colony off the southern tip of the Malay Peninsula, split from the Federation of Malaysia.

Singapore’s founding father, Lee Kuan Yew, wanted to establish a military to defend Singapore, which has a landmass of only 274 square miles, since it was ringed by Muslim nations — as is Israel.

He turned to the Jewish state, through the Israeli ambassador in Thailand, for help. Israel sent a military mission led by Maj. Gen. Rehavam Ze’evi, then deputy head of the military’s operations branch. (Ze’evi was assassinated in Jerusalem by Palestinian gunmen in October 2001.)

Lee insisted on secrecy because he didn’t want to antagonize his Muslim neighbors, Malaysia and Indonesia. The team of Israeli instructors arrived in October 1965. Lee wrote in his 2000 autobiography: “To disguise their presence, we called them ‘Mexicans.’ They looked swarthy enough.”

Today, Singapore’s armed forces, 72,500-strong, are considered one of the most advanced militaries in Southeast Asia.

Iron Dome will be a crucial element in Singapore’s drive to build a defensive shield around one of the world’s biggest and most important ports.

Israel is determining how many batteries of Iron Dome and two other systems that will make up its planned multi-layered missile shield, will be required. Each battery, which costs $50 million, can cover an area of around 60 square miles, which means 13 of them would be needed to cover all of Israel.

The Arrow-2 high-altitude anti-missile system, the only tier to be fully tested and established, cost some $2 billion to develop, largely with U.S. funds.

Israel shunned buying already developed U.S. systems, which would be cheaper. So far, the Defense Ministry has budgeted for one Iron Dome battery but will clearly need several more.

“So why develop such an expensive system, instead of acquiring Raytheon’s cheaper Centurion system?” Intelligence Online asked.

“Some Israeli arms programs are too costly for the local market and are developed principally for export. Iron Dome is a typical example.

“From the outset, Iron Dome was always intended for Singapore, which helped finance its development,” Intelligence Online explained. “Iron Dome will be battle-tested in Israel ahead of export to Singapore at a late date.”

Singapore has bought Israeli weapons systems extensively over the years and Israeli defense companies regularly participate at the annual Singapore Air Show.

Israeli Aerospace Industries, state-owned flagship of Israel’s defense industry, has sold Singapore the Barak-1 naval anti-aircraft system. Rafael’s electronic warfare systems are widely deployed with the Singapore navy.

IAI upgraded the Singapore air force’s old 1960s-era Northrop F-5 Tiger fighters, and with Elbit Systems and Singapore Aerospace won a contract in 1997 to modernize Turkey’s fleet of F-5A/B and NF-5A/B aircraft fighter jets.

Singapore has also acquired unmanned aerial vehicles from Elbit and Rafael for surveillance to enhance maritime security in the Malacca Strait and the South China Sea.

 

Days 2 & 3 of Black Friday

Oddly enough, I don’t find any Amazon lightning offer compelling enough, though I might get this on an impulse buy.

Slow day today, I signed up for an eBates account before shopping on Zinio. Got some reads on the go with my $35 credit.

And then to  Tmart.com to get an iPad Bluetooth keyboard as well as a screen protector. My only worry is it arriving promptly as their warehouse is all the way in US ><

Now twiddling my thumbs for Cyber Monday tomorrow 😀

Day 1 of Black Friday

Yesterday was Black Friday and I got some retail therapy out of it.

First package to arrive after 4 hours was some electronic components from Element14 Singapore for a secret military project I’m intending to embark on, a 512MB Raspberry Pi and a wattmeter. Cost: $84.90

Later PayPal emailed me saying my $9.90 from a Deal.com.sg deal has been refunded cos the merchant went belly up.

I then bought a refurbished iPod nano for my sis cos she wanted it with the Lunatik watch strap, but I canceled it after a day after I saw the 7th gen nano with the $129 EarPods. All for $198, so I’ll prolly buy it next week at Lot 1. I also got myself/my family an iPad 3, 32GB for S$618 only!

Then it’s off to Amazon to get a Kindle Paperwhite for Ryan Nicholas and a cheap pair of earphones.

That just about round up my purchases for today.

 

Quad Rotor UAV

I’m going to get started on my mini UAV project with Renyong. Let’s do this! 😀

http://code.google.com/p/ro-4-copter/wiki/Ro4Copter

http://code.google.com/p/arducopter/

http://sg.element14.com/jsp/bespoke/bespoke7.jsp?bespokepage=e14/common/en/technology-first/applications/robotics/automatic-guided-vehicle.jsp

Toy Safety Regulations
http://www.rcgroups.com/forums/showpost.php?p=1315199&postcount=1

Navigation
http://www.holmea.demon.co.uk/GPS/Main.htm

 

New or Refurbs?

I’m thinking of getting an iPad after seeing Bryant’s Clash of Clans in all its splendour lol.

Refurb iPad with WiFi 32GB costs $618, 64GB costs $728

Refurb iPad with WiFi and 3G costs $788.

New iPad with WiFi 16GB costs $658, 32GB costs $788.

Hmmmmmmph