PUBLIC MARKS with tags linux & cluster

"Coda is an advanced networked filesystem"

This tutorial is based on Falko Timme's tutorial for MySQL Cluster 5.0. It shows how to configure a MySQL 5.1 cluster with five nodes: 1 x management, 2 x storage nodes and 2 x balancer nodes. This cluster is load-balanced by an Ultra Monkey package which provides heartbeat (for checking if the other node is still alive) and ldirectord (to split up the requests to the nodes of the MySQL cluster). In this document I use Debian Etch 4.0 for all nodes. Therefore the setup might differ a bit for other distributions. The two data nodes were x64 to use all of the 8GB RAM. Servers were compiled from source so you should be able to make it running on any platform. The MySQL version I use in this setup is 5.1.24-rc. It's a release candidate, but I wanted to use 5.1 to take advantage of Memory-Disk Based tables. Beginning with MySQL 5.1.6, it is possible to store the non-indexed columns of NDB tables on disk, rather than in RAM as with previous versions of MySQL Cluster.

This tutorial shows how to set up a two-node Apache web server cluster that provides high-availability. In front of the Apache cluster we create a load balancer that splits up incoming requests between the two Apache nodes. Because we do not want the load balancer to become another "Single Point Of Failure", we must provide high-availability for the load balancer, too. Therefore our load balancer will in fact consist out of two load balancer nodes that monitor each other using heartbeat, and if one load balancer fails, the other takes over silently.

voici quelques outils pour travailler efficacement sur un groupe de machines

In this article, I briefly introduce the concepts of Xen virtualization and Linux clustering. From there, I show you how to set up multiple operating systems on a single computer using Xen and how to configure them for use with clustering. I should point out that a cluster implemented in this manner does not provide the computational power of multiple physical computers. It does, however, offer a way to prototype a cluster as well as provide a cost-effective development environment for cluster-based software. Even if you're not interested in clustering, this article gives you hands-on experience using Xen virtualization.

À travers un exemple concret, nous vous proposons de déjouer les pièges de la mise en œuvre d’un cluster haute-disponibilité économique, avec équilibrage de charge et constitué uniquement de deux machines !

Ganeti is a cluster virtualization management system based on Xen. In this tutorial I will explain how to create one virtual Xen machine (called an instance) on a cluster of two physical nodes, and how to manage and failover this instance between the two physical nodes.

Kerrighed is a Single System Image operating system for clusters. Kerrighed offers the view of a unique SMP machine on top of a cluster of standard PCs.

Description (pics included) of a Sony PS3 cluster running Linux at NCSU with useful links to resources for programming on the PS3.

Setting Up An iSCSI Environment On Linux Nowadays, the iSCSI technology is quite popular in the storage world. This article shows an iSCSI demo environment which consists of one Debian Linux host and one Netapp Filer.We try to show the most important features of this protocol. 1. What is iSCSI? It is a network storage protocol above TCP/IP. This protocol encapsulates SCSI data into TCP packets. iSCSI allows us to connect a host to a storage array via a simple Ethernet connection (tape drive). This solution is cheaper than the Fibre Channel SAN (Fibre channel HBAs and switches are expensive). From the host view the user sees the storage array LUNs like a local disks. iSCSI devices should not be confused with the NAS devices (for example NFS). The most important difference is that NFS volumes can be accessed by multiple hosts, but one iSCSI volume can by accessed by one host. It is similar to SCSIi protocol: usually only one host has access to one SCSI disk (the difference is the cluster enviroment). The iSCSI protocol is defined in the RFC3720 document by the IETF (Internet Engineering Task Force).

Setting Up An iSCSI Environment On Linux Nowadays, the iSCSI technology is quite popular in the storage world. This article shows an iSCSI demo environment which consists of one Debian Linux host and one Netapp Filer.We try to show the most important features of this protocol. 1. What is iSCSI? It is a network storage protocol above TCP/IP. This protocol encapsulates SCSI data into TCP packets. iSCSI allows us to connect a host to a storage array via a simple Ethernet connection (tape drive). This solution is cheaper than the Fibre Channel SAN (Fibre channel HBAs and switches are expensive). From the host view the user sees the storage array LUNs like a local disks. iSCSI devices should not be confused with the NAS devices (for example NFS). The most important difference is that NFS volumes can be accessed by multiple hosts, but one iSCSI volume can by accessed by one host. It is similar to SCSIi protocol: usually only one host has access to one SCSI disk (the difference is the cluster enviroment). The iSCSI protocol is defined in the RFC3720 document by the IETF (Internet Engineering Task Force).

Ce document étudie les infrastructures et les différents types et méthodes de haute disponibilité dans un environnement donné. Il présente aussi les différentes solutions en termes logiciels et matériels afin de parvenir à une haute disponibilité. Ce document est une documentation ouverte.

DRBD is a block device which is designed to build high availability clusters. This is done by mirroring a whole block device via (a dedicated) network. You could see it as a network raid-1.

A l"heure où les serveurs d"entreprises doivent stocker un volume croissant de données et assurer une haute disponibilité, il est nécessaire d"imaginer des systèmes de mirroring (miroir) autres que simplement sur des disques durs. La redondance de disques durs (RAID-1, RAID-5) permet déjà d"avoir une bonne résistance aux pannes d"un disque (ou plusieurs si on est en RAID-5). Cependant, si c"est la machine qui tombe (le processeur, l"alimentation, le contrôleur de disque,...), on n"a aucun moyen pour relancer le tout rapidement sans perte de données. Ayant été confronté à cette problématique, j"ai envisagé une solution de RAID-1 au travers du réseau. C"est-à-dire que sur deux serveurs, on a deux partitions (une par serveur) qui sont à tout moment une copie exacte l"une de l"autre. C"est un mirroring (miroir) de partitions à travers une interface réseau.

L"idée générale pour assurer la disponibilité d"un service est de faire fonctionner plusieurs machines (deux au minimum) en même temps. Ces machines forment ce qu"on appelle un cluster et chaque machine est un node du cluster. Chacune des machines va vérifier si les autres répondent toujours en prenant le pouls de chacune des autres. Si une machine cesse de fonctionner, les autres assurent le service. Une fois le cluster configuré, on y accède au travers d"une seule et unique adresse IP qui est celle du cluster; qui lui-même est composé de plusieurs nodes. Pour pouvoir mettre en place ce genre de technique, nous allons utiliser l"application HeartBeat qui va se charger de surveiller les machines et d"appliquer une série de scripts définis par l"utilisateur si cela s"avère nécessaire (c"est-à-dire si une machine tombe).

This is only ONE way to keeps nodes in synchronization for a high availability cluster. It suits our needs and may be a good starting block for others so I'm sharing it. Your mileage may vary. We used this on a web server to update ~1GB worth of documents in sync every 10 minutes. Depending on how quickly your files change, you may be able to update more data, more frequently. I would not recommend using this without a private fast ethernet channel, i.e. an extra NIC in each node connected via a crossover cable.