NGrid
Open source grid computing

Ngrid Weblog

December 2, 2005

NGrid poster at NIPS’05

Filed under: Events — Joannes Vermorel @ 5:12 am

deep sea fishing destin fl
new york enlarged male breast surgery
wonder woman naked cartoon
camping sex
drunk horny girls
bachelorette party blowjob
girl give me head
caught babysitter
monster cock anime
cfnm mpeg
wifey blowjob
free horse sex clips
student apartment search
squirt lessons
petite teen thumbs
male nude beach
denim tight
extreme creampies
military trader magazine subscriptions
cartoon bukkake
military boys pics
young nude girls
sex with teacher
key to oral communication
latex dominatrix
tickling teen feet
tits wmv
new found glory sonny mp3
sex flexible
college coeds in thongs
massive female bukkake
12 year old models in thongs
forced cum swallow
celebrity male cocks
black college girls in thong
facial video preview
naughty nurses big boobs
brigitte quinns sexy legs
hot lezzies riding strapon
ponytail fetish
femdom wife and slave husband story
grannies stripping
alyssa doll camel toe
sexy lesbians using strapon
femdoms
sex showers
shower twinks
blowjob babes
amateur double blow job
face fuck gag
laser tattoo removal michigan
Latina Anal Heartbreakers
learn how to deep throat
dp anal
Double Bubble Booty-2 CD-1
cfnm medical exam
hard hat stickers
shave gels
butt licking lesbians
milf group sex
sandra teen micro bikini
black teen masturbating
girls sucking cock through gloryhole
mature porn pics
cheerleader porn lesbian
teachers pet hentia
asia brazil whore cum bitch fuck
i caught mom cheating on dad
hot blonde chicks getting fucked
free giant dildo girls
gallery of shamrock tattoos
delta airline’s secretary
sex orgy movies
whore gag
Other Peoples Honey
Oversized Boobies-2 CD-1
bondage sex rape
hello kitty gothic lolita
bbw cumming
anal cream pie
milwaukee county zoo ala carte
xxx passwords xxx
free male celebrity sex tapes
sadie belle behind gags
schoolboy spanking
gay mature fuck
christie’s toy box
twink deep throat
brunette outdoors solo
dirty debutante ed powers
bj machine
nude athletes
sunday school teacher appreciation poem
sex hard
tranny fucks guy
pure handjob
hung ebony shemales
petite fisting
asian girl feet
amateur adult film
girl using toys
teen fuck party
breast of bollywood actresses
dog lover license plate frames
sex strangulation clips
gallery sexy asian lingerie sexy
virgin fuckers
girls tied in bikini
sex and candy
first anal teen
The MILF Chronicles CD-1
kristina abernathy legs
jamaican pussy
free galleries porn
shemale best sex
brunette squirter
rocky boot clearance
age pee mp3
women in self bondage
ebony spankings
banging in public
apartamentos en granada
petite cum girl
hairy pussy closeup
skinny black girls
lg flatron
hunt realty buffalo
hot sex scenes
hardcore interracial sex movies
exotic bitches
enormous hard ons
redheads nude
teen chat avenue
big butt blondes
amd assembly compare and swap code
hard money loan personal unsecured california ca
mil hun
bikini sexy
oriental strippers nude picture gallery
paying off student loans
animals gay pics
big white butts
Lethal Latinas-3 CD-1
slutty teen cheerleader
fat black fuck
cum swallow stories
petite teen twins
fff licking orgy
love making techniques
vintage fuck
boss sex in office
big titted midgets
personal fitness trainer jobs tampa florida
gay dick suck
brunette fucking
cfnm handjob movie galleries
huge strapon
fuck bleeding cunt
indian naked
crack password for naked news

November 23, 2005

Another disk cache tool, db4o

Filed under: .Net, NGrid.Superfluid — Joannes Vermorel @ 11:03 am

NGrid relies on custom disk management (see Superfluid notes). For now, the needs are fairly simple, NGrid basically requires a persistent hashtable. Several third party tools have been considered such as Microsoft Sql Server, or the Berkeley DB.

An Database For Objects (db4o) implementation has just been added to NGrid.Infrastructure.Cache and should be available in the next release. db4o comes as a really nice 100% managed single .Net 2.0 assembly. Moreover, the assembly is weighting roughly 800ko. If you’ve never heard of totally object-oriented transactional databases, check-out db4o, it’s worth some attention.

November 19, 2005

Distributed Garbage Collector: implementation ideas

Filed under: NGrid.Core, .Net, NGrid.Plasma — Joannes Vermorel @ 2:47 am

NGrid provides a garbage collected programming framework. When the physical grid is distributed, such as NGrid.Plasma, this property involves the implementation of a Distributed Garbage Collector (DGC). Actually, .Net Remoting provides a lease-based DGC; but the lease semantic is very different from the reachability semantic of the Local Garbage Collector (LGC). A lease-based DGC is very interesting tool to for web applications, but is virtually useless for distributed algorithms.

DGC has been a long standing problem in computer science. Basically, implementing an asynchronous yet complete DGC is a challenging task. The asynchronicity property states that no steps should requires more than a two-machine synchronization mecanism. The completeness property states that all the garbage (but only the garbage) should collected. If completeness is not a requirement, then simple solutions exist for DGC such as the reference listing (implemented by Java). Although, reference listing cannot detect distributed cycles of garbage (i.e. two dead objects referencing each other will not be collected).

In NGrid.Plasma v0.2, the acyclic GC is performed by a classical reference listing algorithm. Additionally, we have implemented a complete DGC based on a recent research paper of Veiga and Ferreira. Our implementation differs from the original paper because we did not want our DGC to require any modification either from the .Net LGC or the .Net Remoting (isolation property). The provided DGC is totally managed component built on top of LGC and .Net Remoting.

Designing a totally isolated DGC requires a few additional elements. Intuitively, our DGC implementation relies on a process that we call object inhumation. Basically, an heuristic selects the candidates for cyclic garbage, and those candidates are inhumed: the object is serialized and discarded (only the serialized version is kept). Referenced objects can, in turn, be inhumed too. If the DGC concludes that the object is trully garbage then the object is collected; otherwise (the heuristic guess was wrong), the object can be resurrected anytime. The big advantage of the inhumation method is its potential promptness: rather than waiting to the DGC execution to confirm than a suspected object is indeed garbage, the object can be directly stored on disk (freeing physical memory meantime). In NGrid.Plasma v0.2, the serialized copies are kept in memories, in the next version, they will be kept on disk (like we do for NGrid.Superfluid).

November 18, 2005

NGrid released, Core v0.6, Plasma v0.2. Superfluid v0.1.1

Filed under: NGrid.Core, NGrid.Plasma, NGrid.Superfluid — Joannes Vermorel @ 11:40 am

NGrid.Core is an abstraction layer for grid computing written in C#. The NGrid.Core framework provides a transparent multithreaded and garbage collected programming model. The version v0.6 does not break compatibility with client code but bring a few simplifications on the side of the physical grid API.

NGrid.Plasma is a P2P grid layer associated to NGrid.Core. Plasma targets CPU intensive applications. The version v0.2 includes many bug fixes and features a new Asynchronous Complete Distributed Garbage Collector.

NGrid.Superfluid is an other grid layer. Superfluid targets memory intensive applications (especially when the memory required by the application going beyond the available physical memory). The version v0.1.1 now supports Microsoft Sql Server 2005, SleepyCat Berkeley DB or simply the filesystem to perform disk caching.

Learn more about NGrid:
http://ngrid.sourceforge.net/ (comments)

November 13, 2005

A step further: Quantum bugs

Filed under: .Net, NGrid.Plasma — Joannes Vermorel @ 5:34 am

Definition: quantum bugs disappear when observed too closely.

The development of concurrent/distributed applications can be quite troublesome in terms of debugging. For example, with NGrid.Plasma, I have encountered a subtile bug in the code related to the distributed object locks. The bug was related to unfrequent race conditions. When the application was running under the debugger, the race condition was not encountered (maybe encountered even less frequently) and the application seemed to be working fine. The bug was only happening when the application was not run under the debugger.

Excepts for proof-reading the code line by line, I do not see much options to fight quantum bugs.

November 4, 2005

GObjects: how much do they cost?

Filed under: NGrid.Core, .Net — Joannes Vermorel @ 1:31 pm

NGrid programming relies heavily on RealProxyies provided by the .Net framework. Since NGrid aims to speed-up intensive applications, one can wonder what are the costs involve in the use of GObjects.

Except for the the NGrid.Zero physical grid, other implementations such as NGrid.Plasma or NGrid.Superfluid rely on transparent proxies to intercept all method calls that target a GObject. The costs are basically twofold: memory overhead and CPU overhead. Those overheads are not independent since memory overhead involve cache miss and therefore additionnal CPU latency.

The following memory footprints have been obtained on a 32bit laptop under .Net 2.0:

  • System.Object: 12 bytes
  • RealProxy: 64 bytes (including associated transparent proxy)
  • GObject: 28 bytes (System.Object plus a System.Guid)
  • ReaderWriterLock: 44 bytes

For a distributed physical grid such as NGrid.Plasma, the overhead per GObject should be expected to range from on hundred to two hundred bytes per element. Such overhead is not dramatic, but it should be noticed that distributed objects are much more expensive than regular objects. As a rule of thumb, distributed objects call for fat distributed objects holding more than 1K bytes of regular objects.

November 3, 2005

NGrid presented at NIPS’05

Filed under: NGrid.Core, Events — Joannes Vermorel @ 2:34 am

NIPS is one of the most well known international conference on Machine Learning. The NGrid project will be presented as a poster in the Large Scale Kernel Machines Workshop. Indeed, we are currently working on improving the performance of the Sequential Minimization Algorithm (this algorithm is very useful for Kernel Machines) with NGrid.

November 2, 2005

NGrid blog refactored

Filed under: website — Joannes Vermorel @ 1:30 am

The previous design was basically user-hostile (unreadable, ugly, no navigation). The blog has been refactored very few modifications though.

  • Switching back to default WordPress style.
  • Header added at the top of page.
  • The sidebar is now dedicated to newcomers.

Nevertheless, I think it’s a considerable improvement over the previous version.

October 23, 2005

Using SQL Server as disk cache

Filed under: .Net — Joannes Vermorel @ 1:48 am

The physical grid NGrid.Superfluid includes a custom disk cache management system. Such system relies on two components: a set of heuristics to decide what should be put on disk, and a persistent (aka disk-based) hashtable. In this post, I will discuss on possible persistent hashtable implementation via SQL Server.

Note that the initial release of NGrid.Superfluid was limited to Berkeley DB. The incoming release will include several persistent hashtable implementations based on various external tools (Berkeley DB, .Net Firebird, Microsoft SQL Server) in order to fit the various platforms.

The most natural choice to implement a persitent hashtable in .Net is to wrap the Berkeley DB because Berkeley DB is basically a persistent hashtable by design whereas the other tools are complete relational databases.

Nevertheless, in this post, I present the SQL Server-based persistent hashtable because it’s certainly the most simple and neat implementation. It’s also a nice illustration of basic SQL syntax.


using System;
using System.Data;
using System.Data.SqlClient;
using System.IO;

namespace NGrid.Superfluid
{
public class SqlCacheTable : IDisposable
{
bool isDisposed;
SqlConnection connection;
IFormatter formatter;

public SqlCacheTable(string connectionString)
{
    connection = new SqlConnection(connectionString);
    connection.Open();

    InitializeTable();
    InitializeFormatter(); // snipped
}

private void InitializeTable()
{
    // Table creation
    string createTable =
"CREATE TABLE SUPERFLUID "
   + "(ID UNIQUEIDENTIFIER, DATA IMAGE);";

    SqlCommand command =
        new SqlCommand(createTable, connection);
    command.ExecuteNonQuery();
}

public void Dispose()
{
    if (!isDisposed)
    {
        isDisposed = true;

        string dropTable = "DROP TABLE SUPERFLUID;";

        SqlCommand command =
            new SqlCommand(dropTable, connection);
        command.ExecuteNonQuery();

        connection.Dispose();
    }
}

The constructor of SqlCacheTable assumes implicitely that a SQL database already exists. Based on the provided connection string, we connect to this database and, in the method InitializeTable we create a table SUPERFLUID with two columns named ID and DATA. Note that the .Net corresponding type equivalent to SQL_UNIQUEINDENTIFIER is System.Guid, similarly SQL_IMAGE is associated to byte[]. At disposal, the table SUPERFLUID is deleted.

The table creation/deletion behavior fits the disk cache requirement of NGrid.Superfluid. If the objective was to implement of persistent hashtable, such behavior would requires to be adjusted.


 public void Add(Guid key, object value)
{
    MemoryStream stream = new MemoryStream();
    formatter.Serialize(stream, value);

    string insertObject =
@"INSERT INTO SUPERFLUID VALUES (@Id, @Data);";
    SqlCommand command =
        new SqlCommand(insertObject, connection);

    command.Parameters.Add(
        @"@Id", SqlDbType.UniqueIdentifier).Value = key;
    command.Parameters.Add(
        @"@Data", SqlDbType.Image).Value = stream.ToArray();

    command.ExecuteNonQuery();
}

public object GetItem(Guid key)
{
    string selectObject =
        "SELECT DATA FROM SUPERFLUID WHERE ID = @Id;";
    SqlCommand command =
         new SqlCommand(selectObject, connection);
    command.Parameters.AddWithValue("@Id", key);

    MemoryStream stream = null;

    using (SqlDataReader reader = command.ExecuteReader())
    {
        if (!reader.Read())
throw new InvalidOperationException(
            "#E00 Key no present in DB.");

        byte[] buffer = (byte[])reader.GetValue(0);
        stream = new MemoryStream(buffer);
    }

    return formatter.Deserialize(stream);
}

public void Remove(Guid key)
{
    string deleteObject =
        "DELETE FROM SUPERFLUID WHERE ID = @Id;";
    SqlCommand command =
        new SqlCommand(deleteObject, connection);
    command.Parameters.AddWithValue("@Id", key);

    command.ExecuteNonQuery();
}
}
}

The three methods Add, GetItem and Remove illustrate basic SQL commands. Note that the MemoryStream is used as an intermediate between the original object and its byte[] serialized version.

As a final word, the .Net ObjectSpaces technology, whenever available (not in VS2005 anyway), might provide an even simpler design.

October 21, 2005

Fault tolerance and NGrid

Filed under: NGrid.Core — Joannes Vermorel @ 12:48 am

Some people did ask me what was the current statut for fault tolerance in NGrid. The most current accurate answer is none. But one should keep in mind that a NGrid application is the cunjunction of three elements: a client application, the NGrid.Core layer and a physical grid. Therefore the fault tolerance issue can be treated in different manners within each of those three elements.

Easy and hard ways

The easiest way (for us people developping NGrid) to tackle the fault tolerance problem is at the client application level. That is let the client developper take care of the fault tolerance himself. This option is not totally unreasonable. For example, physical grids like NGrid.Superfluid (running on a single machine) or NGrid.Plasma (running on a cluster) can quite reliable simply if the underlying hardware is.

The hardest way (for us again) is to handle fault tolerance at the physical grid level (no explicit information provided by the client application developper). This option is particularly hard to implement if we intend to maintain the grid performance (fault tolerance can imply a lot of redundancy). Nevertheless for the client developper, it’s the most confortable option because simply says do nothing, we are handling the problem.

Fault tolerance primitives

A third approach is to handle fault tolerance at NGrid.Core level. This approach would mean to add a few fault tolerance primitives in NGrid.Core, as simple and general as possible, and let the client developper rely on those primitives. The difficulty of this approach is not the implementation (whatever primitives are chosen, the burden is pushed toward the physical grids) but rather the choice of the primitives. Indeed, those primitives have to remain, as much as possible, independant from any particular physical grid implementation. Actually, NGrid.Core includes one element that can be viewed as a fault tolerance primitive: the constructor GThread(ThreadStart threadstart, bool isFailSafe). This constructor differs from the System.Threading.Thread ones. The boolean value isFailSafe indicates that the thread can be safely restarted in case of grid failure.

For now, the NGrid project is more focusing on improving the performance of the existing physical grids rather than extending NGrid.Core, but the introduction of good fault tolerance primitives is definitevely something worth considering.

Next Page »