I'm building a web application in lisp + hunchentoot. It's almost
done, except the email part.
I'd like to keep email anonymity, and I really dig craigslist's
system. I assume it works like this:
1. Assign a random email address to a post/ad.
2. When the server receives an incoming email to that address, just
look up the real email address and forward it.
What library do I need to implement this? (I don't think hunchentoot
has an email client. I could be wrong. Never work with email before.)
From: Pascal J. Bourguignon
Subject: Re: Need Help: Setup up a Craigslist-like email forwarding server
Date:
Message-ID: <7cwshtwp21.fsf@pbourguignon.anevia.com>
HalfEmpty <··············@gmail.com> writes:
> I'm building a web application in lisp + hunchentoot. It's almost
> done, except the email part.
>
> I'd like to keep email anonymity, and I really dig craigslist's
> system. I assume it works like this:
>
> 1. Assign a random email address to a post/ad.
> 2. When the server receives an incoming email to that address, just
> look up the real email address and forward it.
>
> What library do I need to implement this? (I don't think hunchentoot
> has an email client. I could be wrong. Never work with email before.)
Search cliki.net for email, and imap, there are several libraries.
In particular, mel-base seems interesting: http://www.cliki.net/mel-base
--
__Pascal Bourguignon__
On Wed, 03 Sep 2008 11:34:30 +0200, <···@informatimago.com> wrote:
> HalfEmpty <··············@gmail.com> writes:
>
>> I'm building a web application in lisp + hunchentoot. It's almost
>> done, except the email part.
>>
>> I'd like to keep email anonymity, and I really dig craigslist's
>> system. I assume it works like this:
>>
>> 1. Assign a random email address to a post/ad.
That means the email server needs to know it is a valid address. Then
you probably need a method to be the local delivery agent.
>> 2. When the server receives an incoming email to that address, just
>> look up the real email address and forward it.
That is a dynamic forwarding scheme, which must work with the above but
you have probably just required a dedicated email server for your
webapp.
>> What library do I need to implement this? (I don't think hunchentoot
>> has an email client. I could be wrong. Never work with email before.)
>
> Search cliki.net for email, and imap, there are several libraries.
> In particular, mel-base seems interesting: http://www.cliki.net/mel-base
Imap is for email clients, OP needs something to talk to a server.
Either CL-SMTP or CL-Sendmail appear to work, they both use the basic
'telnet' tricks for sending email. Thus either can work with any MTA,
with simple modifications. But the above is not a simple email setup.
Those two packages just would cover the backend of your scheme, the
other parts would need to be custom. It's workable, postfix setup is
easy to handle, but you need a friend with an email server background.
---
If you pre-setup the random emailaddy with appropriate .forward files
(makes a lot of clutter) then the situation is simplified. Or perhaps
something via the alias mechanism. These psuedo-static solutions make
more bookeeping and use more disk space, but are simpler than the
dynamic solution.
What happens with your "craigslist" when someone sends an email to
respond? Does that give away the actual email address, or do you deal
with yet another random email address?
--
One of the strokes of genius from McCarthy
was making lists the center of the language - kt
** Posted from http://www.teranews.com **
GP lisper <········@CloudDancer.com> writes:
> On Wed, 03 Sep 2008 11:34:30 +0200, <···@informatimago.com> wrote:
>> HalfEmpty <··············@gmail.com> writes:
>>
>>> I'm building a web application in lisp + hunchentoot. It's almost
>>> done, except the email part.
>>>
>>> I'd like to keep email anonymity, and I really dig craigslist's
>>> system. I assume it works like this:
>>>
>>> 1. Assign a random email address to a post/ad.
>
> That means the email server needs to know it is a valid address. Then
> you probably need a method to be the local delivery agent.
>
>>> 2. When the server receives an incoming email to that address, just
>>> look up the real email address and forward it.
>
> That is a dynamic forwarding scheme, which must work with the above but
> you have probably just required a dedicated email server for your
> webapp.
>
>>> What library do I need to implement this? (I don't think hunchentoot
>>> has an email client. I could be wrong. Never work with email before.)
>>
>> Search cliki.net for email, and imap, there are several libraries.
>> In particular, mel-base seems interesting: http://www.cliki.net/mel-base
>
> Imap is for email clients, OP needs something to talk to a server.
>
> Either CL-SMTP or CL-Sendmail appear to work, they both use the basic
> 'telnet' tricks for sending email. Thus either can work with any MTA,
> with simple modifications. But the above is not a simple email setup.
>
> Those two packages just would cover the backend of your scheme, the
> other parts would need to be custom. It's workable, postfix setup is
> easy to handle, but you need a friend with an email server background.
>
> ---
>
> If you pre-setup the random emailaddy with appropriate .forward files
> (makes a lot of clutter) then the situation is simplified. Or perhaps
> something via the alias mechanism. These psuedo-static solutions make
> more bookeeping and use more disk space, but are simpler than the
> dynamic solution.
>
> What happens with your "craigslist" when someone sends an email to
> respond? Does that give away the actual email address, or do you deal
> with yet another random email address?
I actually replied to this yesterday, but my reply hasn't shown up.
Essentially, GP is right in that this can be best handled at the mail
server level. Most mail servers have the ability to handle exactly what
you need and allowing it to take care of things means you don't have to
worry about all the nitty gritty stuff, like dealing with
temporary/permanent delivery failures, handling bounces etc.
What most of the mail servers lack is a good management initerface and
this is what you could do just in plain old lisp. There are generally a
range of options from the simpler/slower to the faster, but more
complex to setup/manage.
At the low end, you could just use the email alias
functionality. However, this may become too slow if you have a large
number of accounts. In which case, you need to step up to maintaining
various types of transport/address mapping tables (often using some sort
of Berkley DB hash to speed things up). Postfix, sendmail, qmail and
most other mail servers can handle this.
However, there are a number of problems you will need to think
about. You need a scheme to generate unique IDs for the anonymous
address you create, but thats not too hard.
The real problem will be protecting yourself from having the service
abused in various ways and to protect yourself from being used by
spammers or getting your mail server blacklisted. There are many
solutions, but the system begins to get more complex.
You might want to see if you can find some source code for mail
annonymisers - I have seen some in the past and this is essentially what
you are doing. If you can steal the wheel rather than re-invent it, you
will probably have less stress!
Tim
--
tcross (at) rapttech dot com dot au
> The real problem will be protecting yourself from having the service
> abused in various ways and to protect yourself from being used by
> spammers or getting your mail server blacklisted. There are many
> solutions, but the system begins to get more complex.
Thanks. Those are really good points. It sounds like I need to put in
a lot more effort than I expected. Which make me think if it's even
worth it. Maybe I should just not worry about creating fake address
and just use JavaScript to display real email address so bot won't be
able to read it. Not a solid solution, but hopefully it will keep away
the dumber spammers.
On Thu, 04 Sep 2008 19:25:11 +1000, <····@nospam.dev.null> wrote:
> GP lisper <········@CloudDancer.com> writes:
Ah, the abuse side of email, what a joy...
> However, there are a number of problems you will need to think
> about. You need a scheme to generate unique IDs for the anonymous
> address you create, but thats not too hard.
Actually, since you brought this to my attention, the address space is
rather critical to a successful system. If the address space is too
small, a spammer simply picks up the randomized address list and sends
to it -- knowing that most addresses will be reused in a short time.
A large address space is needed, and it better encode a simple
technique to track reuse, or you get yet another data storage
headache. Some more filtering should handle it.
> The real problem will be protecting yourself from having the service
> abused in various ways and to protect yourself from being used by
> spammers or getting your mail server blacklisted. There are many
> solutions, but the system begins to get more complex.
<kenny> only noobs get blacklisted </kenny>
Well the OP described a system of advertising with hidden email
addresses. That is a 1:1 mapping, so it is the same situation as a
normal domain email server. It would be very smart to run any
received-to-be-forwarded email thru a sanitizer, which you should be
doing anyway with any user input.
A fun little problem to think about. I wonder if it's worth doing.
--
One of the strokes of genius from McCarthy
was making lists the center of the language - kt
** Posted from http://www.teranews.com **
GP lisper wrote:
> On Thu, 04 Sep 2008 19:25:11 +1000, <····@nospam.dev.null> wrote:
>> GP lisper <········@CloudDancer.com> writes:
>
> Ah, the abuse side of email, what a joy...
>
>> However, there are a number of problems you will need to think
>> about. You need a scheme to generate unique IDs for the anonymous
>> address you create, but thats not too hard.
>
> Actually, since you brought this to my attention, the address space is
> rather critical to a successful system. If the address space is too
> small, a spammer simply picks up the randomized address list and sends
> to it -- knowing that most addresses will be reused in a short time.
> A large address space is needed, and it better encode a simple
> technique to track reuse, or you get yet another data storage
> headache. Some more filtering should handle it.
A couple ideas:
- If the address space is large enough, collisions should be rare.
They can be handled via "last one wins" and just overwriting the
former alias or more properly by choosing new addresses until an
unused one is found. Hash functions and/or random number generators
may be helpful here.
- As far as reuse goes, one solution is to embed a timestamp in the
email addresses so you can search for old ones to retire. Another
solution is to write daily/weekly/monthly/whatever logfiles tracking
the disposable addresses as they are created. Then simply read these
logs some time delta later and delete all the old address mappings.
Later,
Daniel
On Thu, 04 Sep 2008 21:21:30 -0400, <········@at.tentpost.dot.com> wrote:
>
>
> GP lisper wrote:
>> On Thu, 04 Sep 2008 19:25:11 +1000, <····@nospam.dev.null> wrote:
>>> GP lisper <········@CloudDancer.com> writes:
>>
>>> about. You need a scheme to generate unique IDs for the anonymous
>>> address you create, but thats not too hard.
>>
>> Actually, since you brought this to my attention, the address space is
>> rather critical to a successful system. If the address space is too
>> small, a spammer simply picks up the randomized address list and sends
>> to it -- knowing that most addresses will be reused in a short time.
>> A large address space is needed, and it better encode a simple
>> technique to track reuse, or you get yet another data storage
>> headache. Some more filtering should handle it.
well the other solutions lead to the above headache, but
> - As far as reuse goes, one solution is to embed a timestamp in the
> email addresses
I think it's required. That way the address space can be small
(i.e. the character count), since it is dynamic. The address space
can be sparse too, in order to defeat prediction, and have some
compression/encryption. Then it's decrypt the timestamp, see it if is
valid, and look it up.
--
One of the strokes of genius from McCarthy was making
lists the center of the language - kt
** Posted from http://www.teranews.com **
> > - As far as reuse goes, one solution is to embed a timestamp in the
> > email addresses
>
> I think it's required. That way the address space can be small
> (i.e. the character count), since it is dynamic. The address space
> can be sparse too, in order to defeat prediction, and have some
> compression/encryption. Then it's decrypt the timestamp, see it if is
> valid, and look it up.
Thanks. I like this idea. I will look into it.