Page 1 of 1

Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 8:37 am
by Ankit9148
Hello guys,
I am trying to take vibration data using an accelerometer. I didn't connect any ADC. The accelerometer is taking data at a rate of 400Hz but my pi is stoing it at a different rate. For example, in one seocnd its 250 samples the next second its 270. This is a big problem as I want to have the FFT of the measured signal. To do that I need to have a fixed number of sample per second. Is there anyway to improve this?

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 8:47 am
by PeterO
Welcome to the forums.

Sorry, but you've not provided any real information that will help solve your problem.

What Pi are you using ?
What language is your application written in ?
Post your non-working code and the output that shows the problem.
What is the accelerometer you are using ?
How is it connected to the PI ?

PeterO

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 1:47 pm
by Ankit9148
Thanks for the reply.
I am using Pi 3 B+ with an accelerometer adafruit MMA8451. The language I am using is python. This is the particulart coding portion where my pi is capturing data from accelerometer and writing it to a csv file.

while True:
x, y, z = sensor.acceleration
time_now = datetime.datetime.now().strftime("%Y-%m-%d")
TimePresent = time.time()
Timer = TimePresent - TimeStart

X = x #+ Calcx
Y = y #+ Calcy
Z = z #+ Calcz
count = count + 1
print('DateTime={0} Time ={1} X={2:0.3f} m/s^2 Y:{3:0.3f} m/s^2 Z:{4:0.3f} m/s^2 count={5}'.format(time_now, Timer, X, Y, Z, count))
sensorwriter.writerow([time_now, Timer, X, Y, Z, count])
time.sleep(1/150)
if Timer > TimingA:
exit()

The sampling frquency of the accelerometer is 800Hz. The pi should store 150 datas per second accoridng to the code. But its not storing that many. In addition, the number of samples its storing is different everytime.

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 2:00 pm
by B.Goode
Added forum Code tags to make sense of the Python fragment.
Ankit9148 wrote:
Fri May 17, 2019 1:47 pm
Thanks for the reply.
I am using Pi 3 B+ with an accelerometer adafruit MMA8451. The language I am using is python. This is the particulart coding portion where my pi is capturing data from accelerometer and writing it to a csv file.

Code: Select all

while True: 
                    x, y, z = sensor.acceleration
                    time_now = datetime.datetime.now().strftime("%Y-%m-%d")
                    TimePresent = time.time()
                    Timer = TimePresent - TimeStart

                    X = x #+ Calcx
                    Y = y #+ Calcy
                    Z = z #+ Calcz
                    count = count + 1                  
                    print('DateTime={0}  Time ={1} X={2:0.3f} m/s^2  Y:{3:0.3f} m/s^2  Z:{4:0.3f} m/s^2 count={5}'.format(time_now, Timer, X, Y, Z, count))
                    sensorwriter.writerow([time_now, Timer, X, Y, Z, count])
                    time.sleep(1/150) 
                    if Timer > TimingA:
                        exit()
            
The sampling frquency of the accelerometer is 800Hz. The pi should store 150 datas per second accoridng to the code. But its not storing that many. In addition, the number of samples its storing is different everytime.

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 2:10 pm
by Ernst
Ankit9148 wrote:
Fri May 17, 2019 1:47 pm
Thanks for the reply.
I am using Pi 3 B+ with an accelerometer adafruit MMA8451. The language I am using is python. This is the particulart coding portion where my pi is capturing data from accelerometer and writing it to a csv file.

while True:
x, y, z = sensor.acceleration
time_now = datetime.datetime.now().strftime("%Y-%m-%d")
TimePresent = time.time()
Timer = TimePresent - TimeStart

X = x #+ Calcx
Y = y #+ Calcy
Z = z #+ Calcz
count = count + 1
print('DateTime={0} Time ={1} X={2:0.3f} m/s^2 Y:{3:0.3f} m/s^2 Z:{4:0.3f} m/s^2 count={5}'.format(time_now, Timer, X, Y, Z, count))
sensorwriter.writerow([time_now, Timer, X, Y, Z, count])
time.sleep(1/150)
if Timer > TimingA:
exit()

The sampling frquency of the accelerometer is 800Hz. The pi should store 150 datas per second accoridng to the code. But its not storing that many. In addition, the number of samples its storing is different everytime.
I can not give you a solution but I can make a few comments. The first comment is that you must get used to using [code] tags

Code: Select all

while True: 
                    x, y, z = sensor.acceleration
                    time_now = datetime.datetime.now().strftime("%Y-%m-%d")
                    TimePresent = time.time()
                    Timer = TimePresent - TimeStart

                    X = x #+ Calcx
                    Y = y #+ Calcy
                    Z = z #+ Calcz
                    count = count + 1                  
                    print('DateTime={0}  Time ={1} X={2:0.3f} m/s^2  Y:{3:0.3f} m/s^2  Z:{4:0.3f} m/s^2 count={5}'.format(time_now, Timer, X, Y, Z, count))
                    sensorwriter.writerow([time_now, Timer, X, Y, Z, count])
                    time.sleep(1/150) 
                    if Timer > TimingA:
                        exit()
I must have overlooked to exit condition, can you show me where TimingA is set ?

How much time do you think that the command will take to be executed ? How much time is needed to read the sensor and how much time is needed to print ? Have you timed this part of the code ?
Take some time to think about this and only then read the next lines of this post.

According to you the code should store 150 samples each second, but that can never be reached because of the 0,0066666666666667 second sleep. Theoretically this would be 150 sleeps per second but this ignores the fact that your code needs some time execute, for this reason 150 samples will never be captured.

I would suggest google "python sleep granularity" to find information on the "accuracy" on the sleep command.

I am sure that it can be done somehow but I do not have an answer, maybe somebody else can help.

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 2:24 pm
by Ankit9148
According to your comment, it should be okay if I take time.sleep(1/100). But its not still capturing 100 samples per second. Its like sometimes 70, sometimes 80. The TimingA I am taking is 1 second to understand the output number of samples clearly.
Thank you for your suggestions anyway.

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 2:40 pm
by Ernst
Ankit9148 wrote:
Fri May 17, 2019 2:24 pm
According to your comment, it should be okay if I take time.sleep(1/100). But its not still capturing 100 samples per second. Its like sometimes 70, sometimes 80.
Absolutely correct, working as expected, you have to adjust your thinking.

Consider this, if the time needed to read the data is 0.002 seconds and the time to print is 0.001 seconds then how many reads/prints can done each second ? (answer: 333 and a bit). Now add the sleep 1/100 (or 0.01 sec) for each cycle giving 0.013 sec. That means that in one second about 76.92 transactions can be performed.

I am now going to give you something to consider, you need a different method for timing. I do not have a solution but I would look for a way to wake up the loop using a timer to wake up the task.

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 2:41 pm
by joan
You have to read more carefully.

read sample + store sample + print sample + sleep 0.01 seconds will only do 100 samples per second if read/store/print each take no time. They each take some time. You need to factor in that processing time.

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 2:53 pm
by Ankit9148
joan wrote: You have to read more carefully.

read sample + store sample + print sample + sleep 0.01 seconds will only do 100 samples per second if read/store/print each take no time. They each take some time. You need to factor in that processing time.

Yes, I understand that. But why the number of samples are changing evrytime. Some time its takeing 70 samples, the other time its taking 80 samples. Is there any way that I can have fixed number of samples per second with high precision in case the sampling rate is as high as 100 or 150?

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 3:10 pm
by davidcoton
Ankit9148 wrote:
Fri May 17, 2019 2:53 pm
Is there any way that I can have fixed number of samples per second with high precision in case the sampling rate is as high as 100 or 150?
Here's two to start with:
  1. Record the time before the processing starts. When you get to the sleep, calculate the remaining time to the next sample time and use that.
  2. Use a timer-driven interrupt to trigger the processing.
No doubt there are other ways.

You will have to determine the top sampling rate available in Python (an interpreted language, AKA relatively slow). It may be necessary to use a compiled language (C, C++, etc) for better performance.

You may need to run on an isolated core to avoid system interrupts, which can disrupt timing quite badly.

Ultimately, you may need to use a real time (RT) kernel to allow precise timing.

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 3:12 pm
by Ernst
@davidcotton: you were faster :)

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 5:35 pm
by Paeryn
Even taking into account the time taken to execute the code, as Ernst hinted at earlier time.sleep() doesn't sleep for exactly the specified length time, rather it sleeps for at least* the specified length of time. If, after the specified time has passed, there other processes (especially those with a higher priority than yours) that are also ready to run then your process can be kept sleeping until the Kernel decides that you can have some CPU time.

* If your process receives a signal whilst it is in a time.sleep() then the sleep can return before the specified length of time has elapsed.

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 5:40 pm
by B.Goode
The last sentence in the preceding post answers something that was puzzling me - why does the script store more data than expected... ?
The accelerometer is taking data at a rate of 400Hz but my pi is stoing it at a different rate. For example, in one seocnd its 250 samples the next second its 270.

Re: Why Raspberry pi is storing different number of samples different second?

Posted: Fri May 17, 2019 6:48 pm
by Paeryn
Just noticed PEP 475, as of Python 3.5 time.sleep() won't return early on a signal unless a signal handler catches the signal and raises an exception.

PEP 475 states that all standard library system calls that could terminate early due to interrupts will now (as of Python 3.5) automatically retry with any timeouts adjusted.