Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Email alert images accumulating on sd card

Sat Mar 16, 2019 12:08 pm

Hi, I was using the RPi for a home security system and found that the email alert images were accumulating on the RPi sd card. Is that normal?. How can I receive the PIR triggered email alerts, but without all the individual images accumulating on the card? Thanks.

Andyroo
Posts: 4233
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: Email alert images accumulating on sd card

Sat Mar 16, 2019 12:24 pm

Bit baffled by your terminology :oops:

Do you mean images the should be attached to emails are staying on the Pi? If so that could be program bug as images are sent with the email as a copy and the program should be able to delete the after sending the mail or it could be correct to act as an archive to show you what happened.

It would help if you detailed your set up a little more and let us know the programs you are using to detect movement, create images and send emails as a min. (Please see other threads for the great crystal ball disaster) :lol:
Need Pi spray - these things are breeding in my house...

Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Sat Mar 16, 2019 12:59 pm

That's right. The images were getting sent to my email and , it seems, also accumulating on the sd card.
With the last python code and general setup, I was getting "out of resources" messages that I couldn't fix. So I decided to format the sd card and start from scratch.
My plan is to keep it as simple as possible. I want to run a python script that will send me email alerts with a few images every time the PIR detects movements.
Mainly, I'd like to know how to prevent the images accumulating on the the 16 GB sd card, because I think a large number of images might possibly cause an "out of resources" message.
Just now, I have only the new NOOBS os installed on the card. Nothing else yet.I'm using the Rpi 3b+. Thanks

Andyroo
Posts: 4233
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: Email alert images accumulating on sd card

Sat Mar 16, 2019 1:13 pm

Kill NOOBS - it causes more issues than it fixes and needs a refresh. Just use the Raspbian images or PINN. If you use Balena Etcher you do not even need to unzip AND it runs a verify.

No need to format the SD card - just wears them out quicker and Etcher overwrites everything it finds :D :o :D

To send an email from Python you can use a few methods (directly / via a command line etc) but this article covers setting up a Gmail account for development and sending mails. It covers sending the pictures under the 'Adding Attachments Using the email Package' - you may want to shrink them depending on your camera resolution.

Once you have sent the mail then deleting the file is simply done with the os.remove('picture.jpg') command.
Need Pi spray - these things are breeding in my house...

Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Sat Mar 16, 2019 1:20 pm

I'm using Linux mint and etcher. Though I hope PINN will be reasonably user friendly. Oh! You mean I needn't format the card, just let Etcher wipe everything and install PINN, right? Thanks

Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Sat Mar 16, 2019 2:01 pm

BY the way, this is the error I was getting with my previous setup:

Code: Select all

pi@raspberrypi:~ $ python alarm.py
mmal: mmal_vc_port_enable: failed to enable port vc.null_sink:in:0(OPQV): ENOSPC
mmal: mmal_port_enable: failed to enable connected port (vc.null_sink:in:0(OPQV))0x15585e0 (ENOSPC)
mmal: mmal_connection_enable: output port couldn't be enabled
Traceback (most recent call last):
  File "alarm.py", line 62, in <module>
    camera = picamera.PiCamera()
  File "/usr/local/lib/python2.7/dist-packages/picamera-1.13-py2.7.egg/picamera/camera.py", line 433, in __init__
  File "/usr/local/lib/python2.7/dist-packages/picamera-1.13-py2.7.egg/picamera/camera.py", line 513, in _init_preview
  File "/usr/local/lib/python2.7/dist-packages/picamera-1.13-py2.7.egg/picamera/renderers.py", line 558, in __init__
  File "/usr/local/lib/python2.7/dist-packages/picamera-1.13-py2.7.egg/picamera/mmalobj.py", line 2212, in enable
  File "/usr/local/lib/python2.7/dist-packages/picamera-1.13-py2.7.egg/picamera/exc.py", line 184, in mmal_check
picamera.exc.PiCameraMMALError: Failed to enable connection: Out of resources
And could you tell me how I could alter this python code (which I think is from this site: https://circuitdigest.com/microcontroll ... ert-system ), so that after each email alert image is sent, I can automatically delete the image from the sd card? Thanks.

Code: Select all


#!/usr/bin/env python3


import RPi.GPIO as gpio
import picamera
import time

import smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEText import MIMEText
from email.MIMEBase import MIMEBase
from email import encoders
from email.mime.image import MIMEImage
 
fromaddr = "email_address"    # change the email address accordingly
toaddr = "to_another_email_address"
 
mail = MIMEMultipart()
 
mail['From'] = fromaddr
mail['To'] = toaddr
mail['Subject'] = "Attachment"
body = "Please find the attachment"

led=17
pir=18
HIGH=1
LOW=0
gpio.setwarnings(False)
gpio.setmode(gpio.BCM)
gpio.setup(led, gpio.OUT)            # initialize GPIO Pin as outputs
gpio.setup(pir, gpio.IN)            # initialize GPIO Pin as input
data=""

def sendMail(data):
    mail.attach(MIMEText(body, 'plain'))
    print data
    dat='%s.jpg'%data
    print dat
    attachment = open(dat, 'rb')
    image=MIMEImage(attachment.read())
    attachment.close()
    mail.attach(image)
    server = smtplib.SMTP('smtp.gmail.com', 587)
    server.starttls()
    server.login(fromaddr, "make_password")
    text = mail.as_string()
    server.sendmail(fromaddr, toaddr, text)
    server.quit()

def capture_image():
    data= time.strftime("%d_%b_%Y|%H:%M:%S")
    camera.start_preview()
    time.sleep(5)
    print data
    camera.capture('%s.jpg'%data)
    camera.stop_preview()
    time.sleep(1)
    sendMail(data)

gpio.output(led , 0)
camera = picamera.PiCamera()
camera.rotation=0
camera.awb_mode= 'auto'
camera.brightness=55
while 1:
    if gpio.input(pir)==1:
        gpio.output(led, HIGH)
        capture_image()
        while(gpio.input(pir)==1):
            time.sleep(1)

    else:
        gpio.output(led, LOW)
        time.sleep(0.01)


Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Wed Mar 20, 2019 10:32 pm

I should have explained that when I get an email alert from the PIR sensor, that email contains a few images from that particular alert BUT also contains other images which I had received in previous emails, from previous PIR alerts. So this leads to receiving emails with lots of "backdated" alerts. Any idea what may be causing this? Thanks

Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Thu Mar 21, 2019 2:08 pm

Thanks Paul, for that suggestion:

Code: Select all

 subject='Security allert!!'
        msg = None
        msg = MIMEMultipart() 
I also thought of trying the line earlier in the code.
I will let you know how it goes, thanks!

And sorry for duplicating the thread!

Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Wed Mar 27, 2019 7:31 am

Hi, I have tried the following script, but the result is that no email is sent due to the line;
:

Code: Select all

msg = None 

Code: Select all

        subject='Security alert!!'
        msg = None
        msg = MIMEMultipart()  
and this is the same in a more complete context:

Code: Select all

while True:
    if GPIO.input(18):
        print("Motion...")
        #camera warm-up time
        time.sleep(2)
        P.capture('movement.jpg')
        time.sleep(10)
        subject='Security alert!!'
        msg = None
        msg = MIMEMultipart()
        msg['Subject'] = subject
        msg['From'] = me
        msg['To'] = toaddr

        fp= open('movement.jpg','rb')
        img = MIMEImage(fp.read())
        fp.close()
        msg.attach(img)
        server = smtplib.SMTP('smtp.gmail.com',587)
        server.starttls()
        server.login(user = 'from@gmail.com',password='*********')
        text=msg.as_string()
        server.sendmail(me,toaddr,text)
        server.quit()  
Now I'm wondering whether this accumulating message thing may be normal in gmail by default, and whether it may be possible to alter that configuration in gmail settings, so that previous messages are not included. I notice that if I exchange, say 20 different emails with someone, by the time I get the 20th mail, all the previous mail messages are included, which can make it harder to find the most recent message. Any thoughts on that? So it's similar with the home security email alerts? Any ideas how to solve this?. Thanks

Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Wed Mar 27, 2019 8:49 am

Just to clarify:
Is there a way to configure my Gmail inbox, or alter the python script, so that I always receive only one message at a time from the same sender? I mean, to make it the same as sending and receiving letters by post. Each sent letter only contains one message, not a record of previous conversations that I have already read.

I also tried the "msg = None" earlier in the script, but the mail still gets sent along with the previous messages

Code: Select all

GPIO.setup(18, GPIO.IN)
while True:
    if GPIO.input(18):
        msg = None
        print("Motion...")
        #camera warm-up time  
Thank you.

User avatar
B.Goode
Posts: 8274
Joined: Mon Sep 01, 2014 4:03 pm
Location: UK

Re: Email alert images accumulating on sd card

Wed Mar 27, 2019 9:01 am

{Disclaimer - I haven't rigorously followed the back story of this, and haven't tried this suggestion... }

If you gave each email message a unique Subject line, perhaps Google would find it more difficult to keep them together in a single 'conversation'? Perhaps a timestamp appended to the "Security Alert" string?

Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Wed Mar 27, 2019 9:23 pm

I haven't tried the timestamp idea, but I did change the subject line in the python script, and I think it seems to make no difference.
I say "I think" because now I'm getting the dreaded "out of resources" error, so I can't connect to the camera. But I'm wondering whether this accumulating message issue is compatible with sending email alerts. Or whether, on the other hand, there is a solution to prevent the repeated messages. Thanks.

Paul Hutch
Posts: 380
Joined: Fri Aug 25, 2017 2:58 pm
Location: Blackstone River Valley, MA, USA
Contact: Website

Re: Email alert images accumulating on sd card

Wed Mar 27, 2019 10:59 pm

I have a RasPi0W monitoring the dehumidifier in my basement, it emails me when it needs to be emptied and sends a status email every day at midnight UTC. I haven't had a full email since the beginning of January, winter is dry here, but I do get the daily emails.

There are whole bunch of the daily emails in my Gmail trash and they all have the exact same subject and message text since no events (full, emptied) have happened since the beginning of January. Gmail is not linking them to one another in any way, they are all standalone emails.

The big differences I notice between my code and yours is that yours attaches images and mine spawns a seperate thread to send email. For debugging this issue I'd eliminate the attachment from the emails and see if you then get individual unlinked messages like I get.

Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Thu Mar 28, 2019 8:14 am

Just to add, in case it may be relevant, what I think may have happened, but I'm no expert.
In recent testing and setting up the camera and email alerts, I think the accumulated alerts reached about 56 in the last alert I received. But each image had a resolution of:

Code: Select all

P.resolution= (1024,768) 
I'm wondering whether this may have caused the "out of resources" error. Though, I suppose it could have been something else. I know I could have set a lower resolution, but I assume that would only postpone being out of resources?
I also think the images may be accumulating on the sd card. As regards the general thing in Gmail (I think Hotmail is the same) about including already read messages in emails, I don't understand the reason for it. Or I think it should be possible for users to choose in mail settings whether they want that feature. Or am I missing something? Thanks
Last edited by Dubman on Thu Mar 28, 2019 8:58 am, edited 1 time in total.

Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Thu Mar 28, 2019 8:53 am

Paul, do you mean just making the following change to this script? (See: #Delete this line) The script is from this site:
https://iotbyhvm.ooo/raspberry-pi-home- ... -tutorial/

Code: Select all

from picamera import PiCamera
from time import sleep
import smtplib
import time
from datetime import datetime
from email.mime.image import MIMEImage
from email.mime.multipart import MIMEMultipart
import RPi.GPIO as GPIO
import time

toaddr = 'TO_EMAIL_address'
me = 'FROM_EMAIL_address'
Subject='alert'

GPIO.setmode(GPIO.BCM)

P=PiCamera()
P.resolution= (1024,768)
P.start_preview()
    
GPIO.setup(23, GPIO.IN)
while True:
    if GPIO.input(23):
        print("Motion...")
        #camera warm-up time
        time.sleep(2)
        P.capture('movement.jpg')
        time.sleep(10)
        subject='Security allert!!'
        msg = MIMEMultipart()
        msg['Subject'] = subject
        msg['From'] = me
        msg['To'] = toaddr
        
        fp= open('movement.jpg','rb')
        img = MIMEImage(fp.read())
        fp.close()
        msg.attach(img) #DELETE THIS LINE

        server = smtplib.SMTP('smtp.gmail.com',587)
        server.starttls()
        server.login(user = 'FROM_EMAIL',password='PASSWORD')
        server.send_message(msg)
        server.quit()
        
As soon as I get the script up and running I'll try that and report back. Thanks

Paul Hutch
Posts: 380
Joined: Fri Aug 25, 2017 2:58 pm
Location: Blackstone River Valley, MA, USA
Contact: Website

Re: Email alert images accumulating on sd card

Fri Mar 29, 2019 12:04 am

Dubman wrote:
Thu Mar 28, 2019 8:53 am
Paul, do you mean just making the following change to this script? (See: #Delete this line)
Yes, that should make it send just a regular email and if those stay as separate emails then we can be sure it's an attachment handling issue.

Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Fri Mar 29, 2019 10:16 am

Ok, thanks. I will try that. Regarding an earlier suggestion from Andyroo:
Once you have sent the mail then deleting the file is simply done with the os.remove('picture.jpg') command.
Would this be the correct way to implement that idea: script (with some extra lines I just added) from https://iotbyhvm.ooo/raspberry-pi-home- ... -tutorial/
The lines I added are

Code: Select all

import os 
os.remove('movement.jpg')
print("File Removed!")  

See comments: #NEW LINE

Code: Select all

from picamera import PiCamera
from time import sleep
import smtplib
import time
import os  #NEW LINE
from datetime import datetime
from email.mime.image import MIMEImage
from email.mime.multipart import MIMEMultipart
import RPi.GPIO as GPIO
import time

toaddr = 'TO_EMAIL_address'
me = 'FROM_EMAIL_address'
Subject='alert'

GPIO.setmode(GPIO.BCM)

P=PiCamera()
P.resolution= (1024,768)
P.start_preview()
    
GPIO.setup(23, GPIO.IN)
while True:
    if GPIO.input(23):
        print("Motion...")
        #camera warm-up time
        time.sleep(2)
        P.capture('movement.jpg')  #IMAGE NAME
        time.sleep(10)
        subject='Security allert!!'
        msg = MIMEMultipart()
        msg['Subject'] = subject
        msg['From'] = me
        msg['To'] = toaddr
        
        fp= open('movement.jpg','rb')
        img = MIMEImage(fp.read())
        fp.close()
        msg.attach(img)

        server = smtplib.SMTP('smtp.gmail.com',587)
        server.starttls()
        server.login(user = 'FROM_EMAIL',password='PASSWORD')
        server.send_message(msg)
        os.remove('movement.jpg')[/s]    #NEW LINE
        print("File Removed!")     #NEW LINE
        server.quit()
        
Would that be correct as a way to prevent images staying on the sd card? Although I suppose it would make no difference to the Gmail previous mails accumulating in newly sent mails? Thanks

Dubman
Posts: 122
Joined: Thu Mar 15, 2018 3:53 pm

Re: Email alert images accumulating on sd card

Tue Apr 02, 2019 6:48 pm

Thanks for all the advice. Just now I'm looking into other alternatives - using an IP camera or maybe a webcam. I want to see if I can avoid having the Rpi in a plastic box outside the house, due to the short cable on the camera module. Thanks again.

Return to “General discussion”