kira2697
Posts: 8
Joined: Thu Jan 24, 2019 4:13 pm

Some random error

Sat Jul 13, 2019 10:45 am

I am collecting data from different sensors using Rpi working perfectly getting the output as expected as saving it to an excel sheet, this is the 1st part. now I wrote the second program which reads that excel sheet and plots the data using pandas and matplotlib, and this creates some random error to what I feel is that at the same time these two programs want to access the excel file and that create an error in the second program. to handle it I made one single program in multithread and instead of getting data from the excel I am making a list of latest 10 which is used to plot, but now I am getting no error but only my 2nd thread is working and not the 1st one. I checked basic multithreading program with my 1st program as 1st and printing out these data in the 2nd thread and it's working fine, as well as a different program wherein 1st threading I gave a list and in 2nd I called matplotlib.animation to plot and it was perfect.

you can find the codes on this link

1st program : https://github.com/Shubham2697/LED_lux_ ... or_data.py
2nd program: https://github.com/Shubham2697/LED_lux_ ... imation.py

merged as multi-thread program: https://github.com/Shubham2697/LED_lux_ ... r/merge.py

Thanks in advance

User avatar
Michiel O.
Posts: 178
Joined: Mon Dec 12, 2016 12:06 pm

Re: Some random error

Sat Jul 13, 2019 1:33 pm

kira2697 wrote:
Sat Jul 13, 2019 10:45 am
this creates some random error to what I feel is that at the same time these two programs want to access the excel file
How did this error manifest itself? Did you get a stack trace? Can you paste the stacktrace?
"You can't actually make computers run faster, you can only make them do less." - RiderOfGiraffes

User avatar
MrYsLab
Posts: 353
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Some random error

Sat Jul 13, 2019 2:06 pm

I think your code structure has lots of issues. I converted your code (untested) to a single class and hopefully this will help you get things going.

Code: Select all

import openpyxl  # read, write excel Sheet
import time  # import time from system
from RPLCD import CharLCD  # Configure LCD
import RPi.GPIO as GPIO  # Configure GPIO pins
import Adafruit_ADS1x15  # Import ADC to Rpi
import board
import busio
import adafruit_tsl2591
import threading
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import queue


class Combined(threading.Thread):
    def __init__(self):
        self.temperature = queue.Queue(10)
        self.voltage = queue.Queue(10)
        self.current = queue.Queue(10)
        self.luxxx = queue.Queue(10)
        self.fig = plt.figure(1)

        self.a11 = 0
        self.acd = []
        self.m = 0
        self.h = 0

        self.a = []
        self.c = []
        self.w = []
        self.x = []
        self.y = []
        self.z = []
        self.lcd = CharLCD(cols=16, rows=2, pin_rs=26, pin_e=19, pins_data=[13, 6, 5, 11], numbering_mode=GPIO.BCM)
        self.lcd.clear()

        # define lux sensor
        self.i2c = busio.I2C(board.SCL, board.SDA)
        self.sensor = adafruit_tsl2591.TSL2591(self.i2c)

        self.adc_value = []
        self.adc = Adafruit_ADS1x15.ADS1115()
        self.GAIN = 1

        self.book = openpyxl.load_workbook('appending.xlsx')
        self.sheet = self.book.active

        self.dim = self.sheet.dimensions
        self.c = list(self.dim)
        self.k = self.c[-1]
        self.t = int(self.k)

        # initialize thread
        threading.Thread.__init__(self)
        self.daemon = True

        # start the second thread
        self.start()

        # kickoff the animate method
        while 1:
            ani = animation.FuncAnimation(self.fig, self.animate, interval=500)
            plt.show()

    def animate(self, i):
        time.sleep(1)

        tem = self.w[-10:]
        volt = self.x[-10:]
        curr = self.y[-10:]
        luxx = self.z[-10:]
        ax1 = self.fig.add_subplot(221)
        ax1.clear()
        ax1.grid()
        ax1.plot(tem)
        plt.ylabel("Temperature in degreeC")
        plt.xlabel("N")
        ax2 = self.fig.add_subplot(222)
        ax2.clear()
        ax2.grid()
        ax2.plot(volt)
        plt.ylabel("Voltage in volt")
        plt.xlabel("N")
        ax3 = self.fig.add_subplot(223)
        ax3.clear()
        ax3.grid()
        ax3.plot(curr)
        plt.ylabel("current in amp")
        plt.xlabel("N")
        ax4 = self.fig.add_subplot(224)
        ax4.clear()
        ax4.grid()
        ax4.plot(luxx)
        plt.ylabel("Intensity In lux")
        plt.xlabel("N")

    def run(self):

        while True:
            print("\n")
            # time
            self.acd = self.time_f(self.a11, self.m, self.h)
            self.a11 = self.acd[0]
            self.m = self.acd[1]
            self.h = self.acd[2]

            adc_value = self.teraadc()
            adc_value.append(self.luxsensor())
            j = 1

            for i in adc_value:
                self.sheet.cell(row=self.t + 1, column=j).value = i
                j = j + 1
                if j > 4:
                    self.t = self.t + 1
                    break
            self.book.save('appending.xlsx')

    def time_f(self, sec, minute, hour):

        sec += 1
        if sec > 59:
            minute = minute + 1
            sec = 0
            if minute > 59:
                hour = hour + 1
                minute = 0
        aaa = [sec, minute, hour]
        print(hour, " ", minute, " ", sec)
        return aaa

    def teraadc(self):
        a = []

        temp = round((self.adc.read_adc(0, gain=self.GAIN) - self.adc.read_adc(3, gain=self.GAIN)) * 5.0 / (32767 *
                                                                                                            .0065), 4)
        print("Temp", temp)
        self.lcd.write_string('Temp: ' + str(temp))
        self.temperature.put(temp)
        self.w = list(self.temperature.queue)
        if self.temperature.full():
            self.temperature.get()
        time.sleep(1)
        self.lcd.clear()

        volt = round(self.adc.read_adc(1, gain=self.GAIN) * 4.8 * 72.2 / 32767, 4)
        print("Voltage", volt)
        self.lcd.write_string(u'Volt: ' + str(volt))
        self.voltage.put(volt)
        self.x = list(self.voltage.queue)
        if self.voltage.full():
            self.voltage.get()
        time.sleep(1)
        self.lcd.clear()

        curr = round(self.adc.read_adc(2, gain=self.GAIN) * 4.6 * 0.95 / 32767, 4)
        print("Current", curr)
        self.lcd.write_string(u'Current: ' + str(curr))
        self.current.put(curr)
        self.y = list(self.current.queue)
        if self.current.full():
            self.current.get()
        time.sleep(1)
        self.lcd.clear()

        a = [temp, volt, curr]
        return a

    def luxsensor(self):
        lux = round(self.sensor.lux * 2, 4)
        print("Lux", lux)
        self.lcd.write_string(u'Lux: ' + str(lux))
        self.luxxx.put(lux)
        if self.luxxx.full():
            self.luxxx.get()
        self.z = list(self.luxxx.queue)
        time.sleep(1)
        self.lcd.clear()
        return lux

ghp
Posts: 1396
Joined: Wed Jun 12, 2013 12:41 pm
Location: Stuttgart Germany
Contact: Website

Re: Some random error

Sat Jul 13, 2019 4:44 pm

Yes, think MrYsLab is correct with the 'many issues'.
- you did not strip down the code to the pure minimum. Not may people in this world do have exactly the same LCD, ADC, lux sensor and can run your code for validation just as provided. I used here

Code: Select all

        adc_value = [math.sin(time.time()/66.666), 22.3, 5, time.time()]
to get some changing values. A sample excel sheet also was missing.
- using excel as a temp buffer makes things complicated. Usually excel are not very efficient. When only the last ten entries are used for display, then use in memory buffers. For long data rows, use database like sqlite or mysql or others.
- your writing procedure to the excel is overwriting the header row "temp", "current",... from time to time. This makes the read procedures crash.
With the help of a max_row parameter in the lib, this problem was corrected.

Code: Select all

        book=openpyxl.load_workbook('appending.xlsx')
        sheet = book.active
        max = sheet.max_row
        print("max", max)
       ...
        j=1
        for i in adc_value:
            sheet.cell(row=max+1,column=j).value=i
- the write procedure should not keep the file open for a long time. This is better:

Code: Select all

    while True:
        ... acquire all the values
        book=openpyxl.load_workbook('appending.xlsx')
        sheet = book.active
        ... save values
        print("save complete")
        book.close()
        time.sleep(0.11) # give some delay only outside the open-close-block.

- when running the update and animation in one multiprocessing code, there is some need for 'do not read while write' aka "a Lock is needed".
Here the basic code layout around this lock.

Code: Select all

        
def collect2(lock_):
    while True:
        pass # do the updates here

lock__ = None
def anim(lock_):
    global lock__
    lock__=lock_  # make the lock available for the animate function. Possibly there is a smarter way, but it works
    while(1): # think the qhile() is not needed here
        print("anim loop")
        ani=animation.FuncAnimation(fig, animate, interval=500)
        plt.show()

if __name__ == '__main__':
    import multiprocessing
    
    lock_ = multiprocessing.Lock()
    p0 = multiprocessing.Process(target=anim, name="A", args=(lock_,) )
    p1 = multiprocessing.Process(target=collect2, name="B",  args=(lock_,))
    p0.start()
    p1.start()

    p0.join()
    p1.join()

I leave it to you to insert the multiprocessing Lock into your code. Did this here, while writing these lines this multiprocess things runs without problems.

Edit: found that the xls file was corrupt after forcing the program to terminate. Shutdown could need some more structure in the code.

kira2697
Posts: 8
Joined: Thu Jan 24, 2019 4:13 pm

Re: Some random error

Sat Jul 13, 2019 8:54 pm

Thank you, everyone, for the replies guys. I know the coding is not perfect :roll: I am a noob with Rpi and python I m learning on the go and I will try these codes and update you guys tomorrow.

Thanks, again.

Return to “Python”