Deadlock probs when changing a barrier

Hi all,

I'm currently writing a multithreaded app (my first, so be gentle!),
which listens on a well-known ****t.  When a client connects to the
****t, a detached thread starts to handle the request.  Each of htese
client threads repeatedly reads from the socket, performs a calculation,
and writes the result of the calculation back to the socket.

The calculation performed is affected by the number of clients, so
I keep track of that.  Each thread uses a pthread barrier, to ensure
that every thread reaches a certain part of the algorithm before
continuing.  A new client can connect at any time, but its presence
can't be allowed to affect any calculation under way.  In other
words, the new client should wait until the end of the current "cycle"
before joining the frey.

Because the number of threads can grow dynamically (it never shrinks),
I need to be able to re-initialise the barrier in a reliable manner.
In the main thread, here's how I'm currently doing this:

num_clients = 0
for (;;) {
    accept incoming connection
    num_clients++
    if (num_clients == 1) {
	pthread_barrier_init (barrier, 1)
    } else {
	lock barrier mutex
	while (barrier.new_cycle == FALSE)
	    pthread_cond_wait (barrier.cv, barrier.mutex)
	
	Obtain barrier write lock
	pthread_barrier_destroy (barrier)
	pthread_barrier_init (barrier, num_clients)
	Release barrier write lock

	barrier.new_cycle = FALSE
	unlock barrier mutex
    }

    create new thread
}


Here's the appropriate part of the client thread:

for (;;) {
    read from socket
    perform calculation
    write result to socket

    obtain barrier read lock
    if (pthread_barrier_wait (barrier) == PTHREAD_BARRIER_SERIAL_THREAD) {
	lock barrier mutex
	barrier.new_cycle = TRUE
	pthread_cond_broadcast (barrier.cv)
	unlock barrier mutex
    }

    release barrier read lock
}


I think the deadlock occurs because of the read/write lock
on the barrier.  It appears that one or more threads grab
a read lock on the barrier, then the main thread grabs a
write lock on it, which prevents further threads from getting
a read lock on it.  This in turn prevents the cycle from
completing, and we deadlock.

I'm hoping that messing about with barriers in this way isn't
too uncommon and that there's an algorithm that works reliably.
Any help or pointers greatfully received!

Cheers,

-- 
Rich Teer, SCSA, SCNA, SCSECA

CEO,
My Online Home Inventory

URLs: http://www.rite-group.com/rich
      http://www.linkedin.com/in/richteer
      http://www.myonlinehomeinventory.com