Ok… currently implementing… so here’s some first remarks/doubts/questions:

1)
$e->get_timestamp() returns the timestamp the message was sent.
How can I get the timestamp when the message was received/pulled? I’m just using time() now, but I believed imap_pop class had this?

2)
What’s the best way to get all mail? Here’s how I’m doing it right now:

Considered a while(get_message()){} instead… but it would probably the same in terms of performance, with the downside of not being able to limit the number of messages?

3)
This is also very time consuming to test/debug because, even if I dont delete the messages, they are never retrieved again. Is there a way to use get_message() without marking them as already retrieved/read by the server?

4)
get_message() accepts two params (used to be a range on previous imap_pop class?)... but delete_and_expunge() takes a range as param? Isn’t that a bit inconsistent? It got me kinda confused at first.

5)
I’m trying to save all email attachments and noticed that save_all_attachments() will save all files, already using fingerprint as the dir (which is perfect!) and will do this for ALL files/attachments with or without disposition. However…get_file_name_array() only seems to show the attachments without disposition!

I could sure use some pointers as to what the best way is to get the filenames, to save on the database, for later easy deletion.

Would also be nice to have these dirs and files set with different chmod/chown. Right now I can only delete them as root and/or by changing their permissions, since fingerprint dirs are 755 and files 644 owned by www-data (only the web server can delete them).

Letting them be executable might be a serious security risk though (I am thinking malicious_script.php and .htaccess attachments). Any ideas?

hi deadelvis,

thanks for the feedback!

1) timestamp:

here is what imap_pop gave you for timestamps (there were 3 values):

peeker_header does not have an internal date that corresponds to date_received_stamp. but, peeker_body does: get_date_pulled().

2) this is the best way to get mail. only feedback is that you could leave out the message count check and just send it get_message(1,50)... peeker figures out the number of messages waiting and will not error if you send it a message end value that is too high (that information should probably be in the docs!).

getting email is VERY slow (as i am sure you know). so, letting peeker get all the messages in an internal loop is faster, but it will consume more memory to load all emails in at once (if that is a concern).

3) yes, testing email is a PAIN because of this. i agree. i considered creating a test structure that would automatically send an email before getting it, but i never got around to that because i moved to an IMAP server. here’s what i do (on IMAP accounts only): set the mailbox to anything not the INBOX. the INBOX is handled in a special way by IMAP servers (doing as you describe, hiding the email after being “read” once).

if you are on a POP server there are no mailboxes, so the best thing to do is to set up a simple mail sender (could do this using CI email class) and have it generate different kinds of emails for testing - that would be a nice utility to offer alongside this code!

4) yes, those two function signatures are not consistent! thank you for pointing that out. i will change that so that delete_and_expunge can accept message range using two parameters (second param optional). i’ll deprecate the old “range style” but i’ll maintain backwards compatibility by checking if there is a colon in the first param. sound good?

5) as you noted, the attachments code is in need of some attention. it works by saving all the files to disk regardless of disposition but then only stores the filename list internally that correspond to actual attachments (not inline imgs with CID tags, specifically, that come as INLINE disposition).

for now, i suggest you use the get_parts_array() method (it’s in the peeker_parts class) - that should have all of the parts information that get_file_name_array() doesn’t. let me know if that works for you. i’m not sure how i want to handle fixing this… maybe just remove the get_file_name_array() method and point to the get_parts_array()?

to manage and delete files… i think the best thing might be for the library to have a way to delete the directory created with the fingerprint and let the webserver still manage the files. as you noted, it currently does not have a way to do that. but, when i was thinking about what the classes should do, i decided to leave that kind of stuff to another set of code (file management) and keep peeker for managing reading/getting/filing/deleting emails from POP and IMAP accounts. that said, i think it would be useful to have it as a function in the main class - something like: delete_local_files($fingerprint). what do you think?

thanks for the great feedback! keep it coming.

Hey Sophistry!

No problem! Thanks for such a great lib! I have now fully replaced imap_pop and implemented the new peeker. It’s thanks to it that mmmmail.com exists.

1) Got it. Doing strtotime($e->get_date_pulled()) as I need a timestamp (maybe a get_timestamp_pulled() would be an idea?)

2) OK. Kept the $message_count though, as I need a way to log how many messages got pulled. Getting mail is indeed very slow, but since I only pull every 2 mins and there isn’t that much mail (at least not yet!)... it’s fine for now.

3) Yeah… total pain. The IMAP tip you suggest is great though. Also found that gmail (where I get my mail from) has a way to reset all POP3 mail under settings so… although it took me quite some clicks for each test, it got the job done.

4) Sounds perfect.

5) Solved it by using the get_parts_array() as you mentioned. A delete_local_files($fingerprint) or…  delete_all_attachments($dir=NULL, $fingerprint) would be awesome. For now I wrote my own, as I need to delete not on fingerprint but on date (older than x days).

Also wrote a get_attachments($fingerprint) which gets the attachments from the filesystem instead, to use after they are saved, and I need to display them next to some mail message.

6) I was also checking the docs for the File class peeker_file.html, but couldn’t really understand how this one would be used. Is it meant for mail attachments? If so, which attachment would $e->get_filename() return when there are more than one on a given email? I tried it briefly but got some errors and decided I get_parts_array() had all the info I needed already anyway.

Cheers

hi deadelvis,

1) added get_timestamp_pulled() and changed internal property to store as timestamp instead of mysql datetime string.

2) don’t know how to speed up mail access - i’ve tried a lot of things. the only real solution would be to use a cache.

3) are you still using POP with the peeker lib? i don’t use POP anymore so i’m curious to hear.

4) made message() and delete_and_expunge() accept two parameters, maintaining BC with imap message range specifier.

5) do you have this code? can you post it? i would like to include this functionality…

6) yes. those docs were wrong. i have updated them in the bitbucket repo.

i also changed a bunch of other things… all listed together here (my commit message)
————————-
added comment about Dependency Injection pattern and link to Martin Fowler article

added get_timestamp_pulled() function

changed internal date handling in peeker_body to use UNIX timestamp

changed function signature of delete_and_expunge() method in peeker class to accept two parameters (start and end ids for messages), or one parameter that is a valid imap range specifier (number in 1:5 colon format or 1,3,5,7 comma separated)

added delete() method in peeker class that mirrors the expunge method

fixed logic in save_all_attachments() method of peeker_parts class (was checking if attachment dir was set, but was doing it in the wrong place, making it impossible to set a custom directory without setting the attachment dir first - even if the custom dir was an absolute path that skirted the need for the attachment dir)

added comment in peeker_parts class suggesting breaking out file system methods into a layer file like the db or listserv layers

changed get_filename_array() method in peeker_parts class to reflect better what it does: gets local filenames that have been re-written to sit better in a filesystem: get_local_file_name_array() is the new name

changed the class property to file_name_array to local_file_name_array to indicate that it holds different filenames (the modified local filenames of the files, (after replacing everything that is not part of this character class [a-z0-9-_.] with underscores, etc..)

removed the has_cid() check from the save_all_attachments() method in peeker_class to allow INLINE attachments to be stored as part of the new local_file_name_array property when calling save_all_attachments

updated peeker_file.php documentation in the docs directory to give accurate usage information

1) 4) 6) Great! Thanks!
2) Can’t really help you there I’m afraid. Right now, and for the requirements, it is very manageable though.
3) Yeah… I am. Works fine.
5) Sure… don’t know how much help it’ll be though. I know the code is not that pretty :( ... it’s quite specific and using local config vars… but I guess it could be repurposed easily.

great! thanks for the feedback on the new peeker class. glad you like it… have you looked at the layers architecture yet? it lets you add methods to the message class to extend functionality without extending the core class. see the autoresponder, listserv, and db methods classes for examples.

also, the detector circuits can help structure message acquisition loops - detectors are another feature of peeker (largely undocumented ATM).

thanks for posting the delete code. i noticed one thing… i suggest putting the helper load command outside of the loop (just a refinement, but it will speed thing up if it doesn’t have to check if the file helper is already loaded on each loop).

i looked into writing and including an attachment deleting method that would be in peeker_parts class but I decided that it would not make sense to be deleting files from the peeker_parts class (or from any part of peeker libs) because the peeker object and its message objects are meant to be a kind of “pipeline” from an IMAP or POP server to the local filesystem. once the “pipeline” is closed, the message objects cease to exist and getting them back (to get the fingerprint for instance) is really the job of whatever system the peeker “pipeline” handed them off to.

in your case, it’s a system that manages files by their age. in another case, it would be a user interacting with the attachments through some kind of database (where the fingerprint and attachment directory would be stored alongside the message).

the only reason i could think of to have a delete_all_attachments() method would be to support the use case where a developer wanted to simply serialize the message objects. when unserialized, the object would provide a mechanism to delete its own attachments.

as of now, i am going to include the delete_all_attachments() even though I am not fully convinced it should be there ... i found this interesting code sample (adapted) at php.net comments for the unlink() function:

No problem. Always a privilege to read, learn from & use code like yours.

I took a quick look at the autoresponder, listserv, and db classes as you suggested… and it made me realize two things:
- How cumbersome my code & coding skill still are…  Thanks.
- That practically the whole site could have been written as just an extension of your peeker message class + detectors.

Didn’t get too deep into it as, to be honest, my brain was already kinda short circuiting.
Maybe if I make the mmmmail.com CI app open source one day (something I just haven’t done for the fear of the responsibility to have to support code I know is not that good/solid, and my ‘n00bishness’ with version control) you’d make the perfect collaborator?

I agree with you… file management is not something that makes much sense for peeker.. and thanks for that helper load tip. A clear overlook.

Best,

PS: I guess your signature is in need of an update… and that wiki imap_pop page in need of a deletion/update?

hey, thanks for the compliments…  you’re too kind. i’ve had my share of code models to learn from too! i’m happy to pass it on.

yeah, detectors (and the declarative programming style they enable) are set up to force you to break up your code into logical discrete parts that can be reused like LEGO bricks in other projects. it’s also easy to divvy up tasks among multiple programmers.

basically, detect and trigger, detect and trigger, detect and trigger.

i updated my sig. thanks for the idea!

Hope it’s ok that I post this question here.  First off, complete virgin coder here, so please forgive any glaring errors.  I need to check a gmail account and save any image attachments.  Fortunately I stumbled upon Peeker!  Don’t know how ‘wrong’ it is, but this code works great for single messages.  I tried to use a for loop and a I tried a while loop - while($this->peeker->message_waiting() to handle multiple messages, but neither are working.  It seems to get stuck when moving on to the second message.  On line 103 where I have

it has saved the current file but for some reason is always looking for the filename from the previous message:  I’ve tried unsetting all the variables as soon as they’ve served their purpose, but that didn’t help.  Tried: then using: but still the same problem.  Any which way I try to loop through multiple messages I get the error above, but the first file is always saved until there is only one message waiting, then I get this error:

But even then, the file gets processed.
So I hope that’s enough to explain my trouble.  4am here and I’m sure I left out something important.  Any suggestions would be terrific.

And I’m only using the view now while I try to get this functioning.  Eventually I’ll have to learn how to use Cron and CI together to have this run hourly, but I’ll fall off that bridge when I come to it.  Had to cut some code to remain under character limit.  Hope I didn’t cut anything critical!
Thanks,
Mark

Hi Mark!

Yes, this is the place to post questions/suggestions about peeker.
EDIT: Better to post messages here so they link to the peeker wiki page discussion. I just realized this is the old imap_pop thread.

Great work for a newbie coder…

Did you verify that the $e->get_parts_array(); method returns the proper list of files? It’s possible that it has a duplicate entry (at least it would explain the issue you have with the same named file).

Also, WRT the looping over messages, I can’t tell if you are successful with that or not. It sounds like you are able to get multiple messages and save their file attachments to the filesystem, but simply have a problem with the last file not being saved and throwing an error.

Let me know.
sophistry

Thanks much for the reply.

The array is working properly.  The trouble I’m having is with looping the function.  Here’s the way I understand it:

The function works great when run using an IF statement to check for waiting messages and grabbing message(‘1’).  All I should have to do to get all messages is either change the IF statement to a WHILE loop or get the message count and throw a FOR loop in after it checks for waiting messages.  Am I wrong here?

Best I can come up with is that there is something wrong with my looping logic.
When I unset all the variables at the end of their loop I get some pretty odd results.

I’ve explained this in more detail here if you’d like to have a look.

Details and screenshots

Mark

hi
m a student. and i got a project where in i have to download pdf from a email.
i tried this library. and enabled imap in gmail account.

at first run i got the following error

pls do help me how to use this library. :-(

imap_pop has been superseded by a better library!

go here:
http://codeigniter.com/forums/viewthread/196550/P15/