ZooKeeper Recipes

• ZooKeeper Recipes and Solutions
• Programming with ZooKeeper - A basic tutorial
• distributed - Zookeeper Producer Consumer - Stack Overflow
• TN4 - Apache Curator - Apache Software Foundation

I am new to Zookeeper and trying to write down some notes about it while I am learning this component. Please skip this article unless you don't mind wasting your time. Thanks.

Introduction

• A Guide to Creating Higher-level Constructs with ZooKeeper
• In this article, you'll find guidelines for using ZooKeeper to implement higher order functions.

I like topic like that.

• All of them are conventions implemented at the client and do not require special support from ZooKeeper
• Hopfully the community will capture these conventions in client-side libraries to ease their use and to encourage standardization.

Several days ago, a friend of mine gave me this link:

• Apache ZooKeeper Made Simpler with Curator (Thanks, Netflix!) | Apache Hadoop for the Enterprise | Cloudera

Not sure whether this is answer to the above suggestion.

• ... that even though ZooKeeper uses asynchronous notifications, you can use it to build synchronous consistency primitives, such as queues and locks. 
• ... because ZooKeeper imposes an overall order on updates, and has mechanisms to expose this ordering.

How can ZooKeeper solve the performance impact for this overall order mechanism?

• attempt to employ best practices. In particular, they avoid 
• polling, 
• timers or 
• anything else that would result in a "herd effect", 
• causing bursts of traffic and 
• limiting scalability.

About herd effect, I found the following explanation very interesting and useful. It really reflects what could happen in distributed programming environment.

Glossary for managing wholes:  H

herd effect - The impact of a excited or concentrated herd of large animals on soil and vegetation. Trampling and hoof action knock down standing vegetation and grind it into soil, along with manure and plant seeds. Herd effect is not stock density -- a large number of animals spread out and walking calmly will not produce herd effect (or much animal impact), whereas a small group of excited animals will. However, some psychological effect of a large, dense herd makes it produce more herd effect than a small herd at the same density. Applied too long or to frequently, herd effect tends to pulverize most soils and cause excessive compaction.

OOB Applications

• Name Service
• Configuration
• Group Membership

• Name service and configuration are two of the primary applications of ZooKeeper. These two functions are provided directly by the ZooKeeper API.

ZooKeeper Performance (Simple test)

I tried to create 10,000 EPHEMERAL nodes, and it took 11 seconds, so it is about 1000 nodes per second. It took 24 milliseconds to list all these 10,000 children. Time to list the children are acceptable, but creating children takes more than I would have expected.

If I use a slower system, a Linux system with two core of
Intel(R) Core(TM)2 CPU         T5600  @ 1.83GHz

It took me 123 seconds to finish creating the nodes.

The create operation will be much faster with asynchronous implementation. It took 1,464 milliseconds to finish creating 10K znodes locally, and 3,384 or even 1,868 milliseconds to delete those nodes. 

Official ZooKeeper Performance Report
Official ZooKeeper Performance Report Measuring Under Various Loads

How to Run ZooInspector

Simply running zooInspector.cmd doesn't work. I did the following things and it works for me:

1. Create a pom.xml file like this

<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemalocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelversion>4.0.0</modelversion>
    <groupid>nonam</groupid>
    <artifactid>noname</artifactid>
    <version>0.0.1</version>

    <dependencies>
        <dependency>
            <groupid>org.apache.zookeeper</groupid>
            <artifactid>zookeeper</artifactid>
            <version>3.4.5</version>
            <exclusions>
                <exclusion>
                    <groupid>com.sun.jmx</groupid>
                    <artifactid>jmxri</artifactid>
                </exclusion>
                <exclusion>
                    <groupid>com.sun.jdmk</groupid>
                    <artifactid>jmxtools</artifactid>
                </exclusion>
                <exclusion>
                    <groupid>javax.jms</groupid>
                    <artifactid>jms</artifactid>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>
</project>

2. run mvn dependency:copy-dependencies
3. run the following command:

java -cp zookeeper-3.4.5-ZooInspector.jar;lib/*;target/dependency/* org.apache.zookeeper.inspector.ZooInspector

Put it into the cmd file if you like.

A very simple ZooKeeper Connection Test Program

Just to demonstrate how to use ZooKeeper API in a very simple way. Works as a HelloWorld program.

Just Try ZK's API

From:

http://www.slideshare.net/scottleber/apache-zookeeper
Slide 11/76


@Test
public void test() throws Exception {
String hosts = "192.168.119.139:2181";
int sessionTimeout = 100;

final CountDownLatch signal = new CountDownLatch(1);
ZooKeeper zk = new ZooKeeper(hosts, sessionTimeout, new Watcher() {

@Override
public void process(WatchedEvent event) {
if(event.getState() == Watcher.Event.KeeperState.SyncConnected) {
signal.countDown();
}
}

});

signal.await();
zk.close();
}

Another HelloWorld from GitHub

https://github.com/wdalmut/zookeeper-helloworld

Need to update the following things:

• dependency for zookeeper. GroupID is org.apache.zookeeper now.
• add maven-compiler-plugin if you need it.

FAQ: NumberFormatException?

• https://jira.talendforge.org/browse/TESB-1325
• http://xulingbo.net/?p=83

In a word: don't use "start"