python multi thread logic bug

JC724

[H]Lite
Joined
Jan 20, 2016
Messages
99
So I have a question I have a project where I will be uses threads to work on databases. So I wrote a small snippet code just to test how the library works. Fairly new to using python and threads. So sometimes the print will print the output correct and other times it will print the output weird.

So I am passing in a list of numbers 2,3,4,9 and one function squares each number and print it and the other cubes each number then prints.

it is supposed to print out mix up of a squared number and a cubed number but sometimes it prints two numbers on the same line. And when I used strings like print('squared number: ', n*n) the letters get all mixed up. I stopped using print('suared number: ',n*n) and instead just use print(n*n). It is easier to see the output

So basically it is not executed the entire print function in the loop of each function.

if it does this in my project it will insert part of the values in the database instead of a full row and cause an error.

So sometimes the output is correct like below:
calculate square numbers
4
9
64
81
calculate cube of numbers
8
27
512
729
('done in : ', 1.6080000400543213)
Done with all the functions now!


Now time with the threads
('done in : ', 0.80765430)
calculate square numberscalculate cube of numbers

4
8
9
27
64
512
81
729

And other times it is like this:

calculate square numbers
4
9
64
81
calculate cube of numbers
8
27
512
729
('done in : ', 1.6069998741149902)
Done with all the functions now!


Now time with the threads
('done in : ', 0.0)
calculate square numberscalculate cube of numbers

84
27
9
512
64
729
81

Actually code below

Code:
#!/usr/bin/python
import psycopg2
import time
import threading


def calc_square(numbers):
    print("calculate square numbers")
    for n in numbers:
        time.sleep(0.2)
        #print('squared number: ',n*n)
        print(n*n)
       
       
def calc_cube(numbers):
    print("calculate cube of numbers")
    for n in numbers:
        time.sleep(0.2)
        #print('cube number: ',n*n*n)
        print(n*n*n)
       

arr = [2,3,8,9]

t = time.time()
calc_square(arr)
calc_cube(arr)

print("done in : ",time.time()-t)
print("Done with all the functions now!")
print("")
print("")
print("Now time with the threads")

s = time.time()
print("done in : ",time.time()-s)

t1 = threading.Thread(target=calc_square,args=(arr,))
t2 = threading.Thread(target=calc_cube,args=(arr,))

t1.start()
t2.start()

t1.join()
t2.join()
 

modi123

[H]ardness Supreme
Joined
Sep 6, 2006
Messages
5,640
That's the point of multithreading. Some processing gets done on one thread, a bit more on the other, etc.

Perhaps pull out your print statements and just add your values to a queue, and print those out when done.


Code:
import time
import threading
import queue


def calc_square(numbers, foo):
    l = []
    for n in range(numbers):
        l.append(n*n)
    foo.put_nowait(l)
       
def calc_cube(numbers, foo):
    l = []
    for n in range(numbers):
        l.append(n*n*n)
    foo.put_nowait(l)

#arr = [2,3,8,9]
n = 100

bar1 = queue.Queue()
bar2 = queue.Queue()


t = time.time()

calc_square(n,bar1)
calc_cube(n,bar2)

print (bar1.get())
print (bar2.get())

print("done in : ",time.time()-t)
print("Done with all the functions now!")

print("")
print("")


print("Now time with the threads")

bar1 = queue.Queue()
bar2 = queue.Queue()

s = time.time()

t1 = threading.Thread(target=calc_square,args=(n,bar1))
t2 = threading.Thread(target=calc_cube,args=(n,bar2))

t1.start()
t2.start()

t1.join()
t2.join()

print (bar1.get())
print (bar2.get())

print("done in : ",time.time()-s)
 
Last edited:

Spun Ducky

Gawd
Joined
Feb 1, 2009
Messages
624
I am not a Python coder but you are having an issue with context switching/race conditions. You need to make sure you use blocking statements and/or thread safe functions.
 

Absalom

Gawd
Joined
Oct 3, 2007
Messages
711
I am not a Python coder but you are having an issue with context switching/race conditions. You need to make sure you use blocking statements and/or thread safe functions.
Precisely what I was thinking. If you don't want your printing mangled (which is what happens when two or more threads compete for the same singular print output), you should write a printing function/method that serializes the act of printing. A queue was mentioned earlier, and that's a form of serialization.

One way to do this is to only allow one thread access to the resource (i.e. the print call) at a time, while blocking the others (they'll get their turn when the resource is released). Python should have synchronization primitives for this.

Some pseudocode would look like this:
Code:
PrintSync( string stuffToPrint )
{
   printLock.Wait();  // wait until resource is available
   print(stuffToPrint);  // use the resource
   printLock.Set();  // release the resource
}
 
Last edited:

Spun Ducky

Gawd
Joined
Feb 1, 2009
Messages
624
Precisely what I was thinking. If you don't want your printing mangled (which is what happens when two or more threads compete for the same singular print output), you should write a printing function/method that serializes the act of printing. A queue was mentioned earlier, and that's a form of serialization.

One way to do this is to only allow one thread access to the resource (i.e. the print call) at a time, while blocking the others (they'll get their turn when the resource is released). Python should have synchronization primitives for this.

Some pseudocode would look like this:
Code:
PrintSync( string stuffToPrint )
{
   printLock.Wait();  // wait until resource is available
   print(stuffToPrint);  // use the resource
   printLock.Set();  // release the resource
}
If I could find it I have a snippet of c# code that has this exact same issue and how to fix it the c# way using locks. I am pretty certain your example would fix the python version without personally knowing python syntax all that well. I have used the c# snippet to teach a few times I may have to dig for it but you got them on the right track.
 

Absalom

Gawd
Joined
Oct 3, 2007
Messages
711
If I could find it I have a snippet of c# code that has this exact same issue and how to fix it the c# way using locks. I am pretty certain your example would fix the python version without personally knowing python syntax all that well. I have used the c# snippet to teach a few times I may have to dig for it but you got them on the right track.
You're not wrong in your thinking. A lock makes perfect sense here. I just didn't want to literally spell out the answer to the OP, so they can figure it out for themselves.
 

gamerk2

[H]ard|Gawd
Joined
Jul 9, 2012
Messages
1,691
The root issue is that "printf" is fundamentally a single-threaded task; whichever of the two threads gets to it first will output it first. This is a classic race condition.

As has been said by others, either you need to put locks around your "printf" statements, or you need to save your thread outputs and execute "printf" in a controlled manner after both threads have finished executing.
 
Top