6 Warning Signs That Your Technology is Headed South | Javalobby
30.6.13
27.6.13
Disk Performance
- Apache Kafka: Why we built this
- How to see disk details like manufacturer in Linux - Unix & Linux Stack Exchange
- Linux File System Read Write Performance Test
- Inspecting disk IO performance with fio | Linux.com
Test on my old Linux PC, with a SATA drive:
Model: "Hitachi HTS54168"
Vendor: "Hitachi"
Device: "HTS54168"
Revision: "SB2O"
Driver: "ahci", "sd"
Driver Modules: "ahci"
Device File: /dev/sda
Size: 156301488 sectors a 512 bytes
Attached to: #16 (SATA controller)
$ dd if=/dev/zero of=zero.bin bs=512 count=1MB conv=notrunc
1000000+0 records in
1000000+0 records out
512000000 bytes (512 MB) copied, 13.8759 s, 36.9 MB/s
Then I tested on a VM:
# dd if=/dev/zero of=zero.bin bs=512 count=1MB conv=notrunc
1000000+0 records in
1000000+0 records out
512000000 bytes (512 MB) copied, 4.3586 seconds, 117 MB/s
# dd if=/dev/zero of=zero.bin bs=512 count=1MB conv=notrunc
1000000+0 records in
1000000+0 records out
512000000 bytes (512 MB) copied, 2.21377 seconds, 231 MB/s
# time dd if=/dev/zero bs=1M count=10K of=dd.out
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 9.63733 seconds, 1.1 GB/s
# time dd if=/dev/zero bs=1M count=10K of=dd.out
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 9.63733 seconds, 1.1 GB/s
Test with a simple Java program:
* PC/SATA:
* Server: 913MB/sec
Is it too fast?
23.6.13
11.6.13
10.6.13
8.6.13
6.6.13
osgix:managed-properties : Managed Properties
- Chapter 10. Compendium Services
- Apache Felix - Apache Felix Config Admin
- ConfigurationAdmin (OSGi Service Platform Release 4 Version 4.2)
- OSGi Alliance Blog: Easier Configuration Admin
- java - How let OSGi (Config Admin) persist/reuse your configuration? - Stack Overflow
- Karaf Tutorial Part 2 - Using the Configuration Admin Service - Liquid Reality - Confluence
5.6.13
Thread Join on Itself
java - Thread join on itself - Stack Overflow
The concept of a thread joining itself does not make sense.
It happens out that the
In other words, the
join() method uses the isAlive() method to determine when to return from thejoin() method. In the current implementation, it also does not check to see if the thread is joining itself.In other words, the
join() method returns when and only when the thread is no longer alive. This will have the effect of waiting forever.I saw this in Helix:
public void run()
{
LockProcess lockProcess = null;
try
{
lockProcess = new LockProcess(clusterName, ZK_ADDRESS,
instanceName, startController);
lockProcess.start();
Thread.currentThread().join();
} catch (InterruptedException e)
{
System.out.println(instanceName + "Interrupted");
if (lockProcess != null)
{
lockProcess.stop();
}
} catch (Exception e)
{
e.printStackTrace();
}
}
Resource, and Rebalancing in Apache Helix
Resources
- Abstraction:
- Helix defines Resources as an abstract concept. Anything could be resources, such as a database, lucene index or a task.
- Resource represents the actual task performed by the nodes. It can be a database, index, topic, queue or any other processing. A Resource can be divided into many sub parts called as partitions.
- Partitionable:
- Each Resource is partitioned into one or more Partitions.
- State
4.6.13
Apache Helix
Apache Helix -
Helix aims to provide the following abilities to a distributed system:
- Automatic management of a cluster hosting partitioned, replicated resources.
- Soft and hard failure detection and handling.
- Automatic load balancing via smart placement of resources on servers(nodes) based on server capacity and resource profile (size of partition, access patterns, etc).
- Centralized config management and self discovery. Eliminates the need to modify config on each node.
- Fault tolerance and optimized rebalancing during cluster expansion.
- Manages entire operational lifecycle of a node. Addition, start, stop, enable/disable without downtime.
- Monitor cluster health and provide alerts on SLA violation.
- Service discovery mechanism to route requests.
3.6.13
1.6.13
ZooKeeper Recipes
- ZooKeeper Recipes and Solutions
- Programming with ZooKeeper - A basic tutorial
- distributed - Zookeeper Producer Consumer - Stack Overflow
I am new to Zookeeper and trying to write down some notes about it while I am learning this component. Please skip this article unless you don't mind wasting your time. Thanks.
Introduction
- A Guide to Creating Higher-level Constructs with ZooKeeper
- In this article, you'll find guidelines for using ZooKeeper to implement higher order functions.
I like topic like that.
- All of them are conventions implemented at the client and do not require special support from ZooKeeper
- Hopfully the community will capture these conventions in client-side libraries to ease their use and to encourage standardization.
Several days ago, a friend of mine gave me this link:
Not sure whether this is answer to the above suggestion.
- ... that even though ZooKeeper uses asynchronous notifications, you can use it to build synchronous consistency primitives, such as queues and locks.
- ... because ZooKeeper imposes an overall order on updates, and has mechanisms to expose this ordering.
How can ZooKeeper solve the performance impact for this overall order mechanism?
- attempt to employ best practices. In particular, they avoid
- polling,
- timers or
- anything else that would result in a "herd effect",
- causing bursts of traffic and
- limiting scalability.
About herd effect, I found the following explanation very interesting and useful. It really reflects what could happen in distributed programming environment.
herd effect - The impact of a excited or concentrated herd of large animals on soil and vegetation. Trampling and hoof action knock down standing vegetation and grind it into soil, along with manure and plant seeds. Herd effect is not stock density -- a large number of animals spread out and walking calmly will not produce herd effect (or much animal impact), whereas a small group of excited animals will. However, some psychological effect of a large, dense herd makes it produce more herd effect than a small herd at the same density. Applied too long or to frequently, herd effect tends to pulverize most soils and cause excessive compaction.
OOB Applications
- Name Service
- Configuration
- Group Membership
- Name service and configuration are two of the primary applications of ZooKeeper. These two functions are provided directly by the ZooKeeper API.
Subscribe to:
Posts (Atom)