DiscussionPlugin support

[ibarton@…]

Hello, email2trac developers.

I'm a developer of DiscussionPlugin for Trac [1] and one of its users
requested an e-mail posting support for it [2]. I see that a good way to
implement this would be to extend your email2trac set of scrips so I'm asking
if would it be acceptable for you to include to mainstream possible patch that
would add DiscussionPlugin support to email2trac or should I fork/reuse
email2trac (or something else) in standalone solution or start to implement
something from scratch?

I haven't studied email2trac code much for now so I don't know how to do it
exactly yet, but in case of possitive answer, I'd digg more into it. Anyway,
any hints or advices would be appreciated.

Thank you for making your stand on this topic.

[1] - https://trac-hacks.org/wiki/DiscussionPlugin
[2] - https://trac-hacks.org/ticket/6883

With best regards,

[bas]

We have already FullBlog? support in email2trac. So it is no problem to add DiscussionPlugin? support as well. The fullblog plugin is not actively maintained so mybe it will be removed again. I do not know al the ins and out of the Discussion plugin so if there are problems with it in email2trac can i send you an email for support?

PS) Just look how the FullBlog? support is added. This is a good starting point. email2trac can handle attachments. I do not know if your plugin supports it.

[ibarton@…]

OK, thank you. I'll start with it at Monday. And yes, DiscussionPlugin? supports attachments.

[ibarton@…]

I've done it but I can't add the diff as an attachment due to spam filter and it's too long to be pasted here. Any suggestions?

[ibarton@…]

OK, I've putted it to my Dropbox: ​http://dl.dropbox.com/u/6882884/discussion_plugin_support.diff . If you have any comments or suggestions, just tell me and I'll change it. I didn't tested it on a "real" e-mail server just from the command line with saved messages. And one another wish, please: If you decide to commit it to the upstream, would it be possible to tag that commit with my nick "Blackhex", just for Ohloh statistics?

[geeksrik@…]

[root@astrotrac email2trac-1.3.2]# patch < discussion_plugin_support-1.diff (Stripping trailing CRs from patch.) patching file email2trac.py.in Hunk #3 succeeded at 300 (offset -5 lines). Hunk #5 succeeded at 1065 (offset -65 lines). Hunk #7 succeeded at 1137 with fuzz 2 (offset -65 lines). Hunk #8 FAILED at 1368. Hunk #9 FAILED at 1475. Hunk #10 succeeded at 1843 (offset -8 lines). Hunk #11 succeeded at 1798 (offset -65 lines). Hunk #12 succeeded at 1881 (offset -8 lines). Hunk #13 succeeded at 1942 (offset -65 lines). Hunk #14 succeeded at 2100 (offset -8 lines). 2 out of 14 hunks FAILED -- saving rejects to file email2trac.py.in.rej

[bas]

Just added for reference

[bas]

I have no problem at all with patching this againt the trunk version

[geeksrik@…]

[root@astrotrac email2trac-1.3.2]# patch < discussion_plugin_support.diff patch unexpectedly ends in middle of line patch: Only garbage was found in the patch input.

[bas]

I have a question about this code:

+               # TODO: Read server base URL from config.
+               # Create request object to mockup context creation.
+               #
+               environ = {'SERVER_PORT' : 80,
+                                  'SERVER_NAME' : 'test',
+                                  'REQUEST_METHOD' : 'POST',
+                                  'wsgi.url_scheme' : 'http',
+                                  'wsgi.input' : sys.stdin}
+               chrome =  {'links': {},
+                                  'scripts': [],
+                                  'ctxtnav': [],
+                                  'warnings': [],
+                                  'notices': []}
+               if self.env.base_url_for_redirect:
+                       environ['trac.base_url'] = self.env.base_url
+               self.req = Request(environ, None)
+               self.req.chrome = chrome
+               self.req.tz = 'missing'

Our trac installation only supports port 443 or does this not matter. Is this code only needed by the Discussion plugin or there also benefits for email2trac?

Is the Discussion plugin only available for trac 0.9:

 self.attach_attachments(context.content_parts, self.VERSION == 0.9)

It is a lot of code ;-). Thanks for the patch. Maybe i will use the Discussion plugin from one of my opensource projects.

[ibarton@…]

geeksrik@…: If you do:

svn co https://subtrac.sara.nl/oss/svn/email2trac
cp discussion_plugin_support.diff email2trac/trunk
cd email2trac/trunk/discussion_plugin_support.diff
patch < d

it should work just fine.

bas: This kind of initialization is just to overcome somehow Trac internals initialization for some stuff with DiscussionPlugin?. I think that the actuall value is not used anywhere in email2trac or DiscussionPlugin? so you can keep it that way. But if I'm wrong, I'll change this to read server name and port from trac.ini.

Is the Discussion plugin only available for trac 0.9:

No, DiscussionPlugin? supports 0.9, 0.10, 0.11 and 0.12. I tested this patch just with 0.11 but it should work with 0.12 as well.

[anonymous]

I had a similar problem with implementing workflow for tickets. I ended up to fake a request:

req = Mock(authname='anonymous', perm=MockPerm(), args={}

Maybe we can do this also for The discussion plugin. So we can delete this code and move this to the function _create_context. This is the only place where self.req is used. Some pseudo code:

req = Mock(authname='anonymous', perm=MockPerm(), args={})
self.req.authname = self.author
self.req.perm = PermissionCache(self.env, self.author)
context = Context.from_request(self.req)

I only asked it, because you use:

 self.attach_attachments(context.content_parts, self.VERSION == 0.9)

It suppose to handle: True or False

[ibarton@…]

I did it that way first, but it's not enough for DiscussionPlugin? since it uses SpamFilterPlugin? that needs request initialized more complexly.

[geeksrik@…]

Ok patching worked now. But when I try running the utility this is what i get:

[root@astrotrac trunk]# /usr/local/bin/email2trac --project=trac < msg.txt

email2trac.py -- Email tickets to Trac.

A simple MTA filter to create Trac tickets from inbound emails.

Copyright 2005, Daniel Lundin <daniel@edgewall.com>
Copyright 2005, Edgewall Software

Authors:
  Bas van der Vlies <basv@sara.nl>
  Walter de Jong <walter@sara.nl>

The scripts reads emails from stdin and inserts directly into a Trac database.

How to use
----------
 * See https://subtrac.sara.nl/oss/email2trac/

 * Create an config file:
    [DEFAULT]                        # REQUIRED
    project      : /data/trac/test   # REQUIRED
    debug        : 1                 # OPTIONAL, if set print some DEBUG info
    trac_version : 0.10              # OPTIONAL, default is 0.11

    [jouvin]                         # OPTIONAL project declaration, if set both fields necessary
    project      : /data/trac/jouvin # use -p|--project jouvin.

 * default config file is : /etc/email2trac.conf

 * Commandline opions:
                -h,--help
                -f,--file  <configuration file>
                -n,--dry-run
                -p, --project <project name>
                -t, --ticket_prefix <name>

SVN Info:
        $Id: email2trac.py.in 356 2010-04-15 13:18:09Z bas $

   No Trac project is defined in the email2trac config file.

this is my conf file in the same folder:

[DEFAULT]
project: /trac-repo
debug: 0
umask: 022
spam_level: 5
reply_all : 0
mailto_link: 0
umask: 022
email_header: 0
trac_version: 0.11
enable_syslog : 1
alternate_notify_template :
alternate_notify_template_update :
drop_spam : 0
verbatim_format: 1
strip_signature: 0
email_quote: >
strip_quotes: 0
ignore_trac_user_settings: 0
black_list: MAILER-DAEMON@
drop_alternative_html_version: 1

[tracproj]
project: /trac-repo
debug: 0
umask: 022
spam_level: 5
reply_all : 0
mailto_link: 0
umask: 022
email_header: 0
trac_version: 0.11
enable_syslog : 1
alternate_notify_template :
alternate_notify_template_update :
drop_spam : 0
verbatim_format: 1
strip_signature: 0
email_quote: >
strip_quotes: 0
ignore_trac_user_settings: 0
black_list: MAILER-DAEMON@
drop_alternative_html_version: 1

[bas]
project: /data/trac/bas
spam_level: 1

[geeksrik@…]

Blackhex, i am adding this query in this ticket itself.

I have a mailbox which accepts emails. Lets say this is "mybox@…", now i want trac discussion plugin to be able to send mails (notificiations) to trac users, and the reply id must be "mybox@…". Assuming someone replies to a discussion, this mail will be left in "mybox@…". A cron job which is running then needs to read this mailbox, route that email to discussion plugin which will put the content at its appropriate place.

Now having said this, I am trying to understand what email2trac does. Firstly I believe this plugin does not scan mailboxes like "mybox@…" (pop or imap). secondly does your diff (patch) actually do this scanning of mailbox? what is the exact cron job i need to run in order that above process happens every minute or two? currently, i am only still getting notifications from DP, but any replies after setting above reply to address does not get picked up by DP.

Can the email2trac plugin provide POP mailbox scanning?

Any help in detailing this a bit will be much appreciated. thanks!

[anonymous]

You don't have properly edited your email2trac.conf. Mine looks like this (when I'm running the script with --project=Test-0.11):

[Test-0.11]
project: /var/lib/trac/Test-0.11
debug: 5
umask: 022
spam_level: 5
reply_all : 0
mailto_link: 0
umask: 022
email_header: 0
trac_version: 0.11
enable_syslog : 0
alternate_notify_template :
alternate_notify_template_update :
drop_spam : 0
verbatim_format: 1
strip_signature: 0
email_quote: >
strip_quotes: 0
ignore_trac_user_settings: 0
black_list: MAILER-DAEMON@
drop_alternative_html_version: 1

[geeksrik@…]

I dont think mine looks any different, but this is what i got:

[root@astrotrac trunk]# /usr/local/bin/email2trac --project=tracproj < msg.txt
Not a valid project name: tracproj
Valid names: [u'bas', 'trac']
[root@astrotrac trunk]#

this is my new config file:

[tracproj]
project: /trac-repo
debug: 0
umask: 022
spam_level: 5
reply_all : 0
mailto_link: 0
umask: 022
email_header: 0
trac_version: 0.11
enable_syslog : 1
alternate_notify_template :
alternate_notify_template_update :
drop_spam : 0
verbatim_format: 1
strip_signature: 0
email_quote: >
strip_quotes: 0
ignore_trac_user_settings: 0
black_list: MAILER-DAEMON@
drop_alternative_html_version: 1

[geeksrik@…]

ok, i figured that out! it was to copied to /usr/local/etc instead of /usr/etc

[root@astrotrac trunk]# locate email2trac.conf
/opt/email2trac/email2trac-1.2.0/email2trac.conf
/opt/email2trac-1.2.0/email2trac.conf
/usr/local/etc/email2trac.conf
[root@astrotrac trunk]# cp email2trac.conf /usr/local/etc/.
cp: overwrite `/usr/local/etc/./email2trac.conf'? y
[root@astrotrac trunk]# /usr/local/bin/email2trac --project=tracproj < msg.txt
Loading environment /trac-repo
[root@astrotrac trunk]#

[geeksrik@…]

Now for the old question: Can the email2trac plugin provide POP mailbox scanning?

[anonymous]

I'm not its developer but as I know it can't. It just imports the file with e-mail message. You'd have to write a simple script that downloads the messages and then executes email2trac on them.

[bas]

Just read this:

• ​https://subtrac.sara.nl/oss/email2trac/wiki/Email2tracMta

And read the Fetchmail setup, maybe this is what you want!

[bas]

There are several discussions in this ticket ;-). But for the SpamFilterPlugin you need this complex setup. email2trac also handles spam it relies on programs that set t a flag in the email header. Maybe we can create a fake request that also init the SpamFilterPlugin. I will do some testing.

[bas]

I have done some little rewriting and moved the statements that where in the class to the function:

• _create_context

So it is easy for me what is mime en what belongs to the discussion plugin and the import statements are moved to the appropriate sectionm see new attachment. Removed also the print statement to stdout that shows which environment was loaded. Most MTA do not like that

[bas]

A new patch for adding support for discussion plugin

[ibarton@…]

I'll check if that doesn't break anyting later...

[ibarton@…]

OK, it didn't.

[bas]

Thanks for testing. Was a bit swamped with a new project for our cluster installation tool. I will apply it asap to the trunk version and then make a new release. I will do some fine tuning that the regex for blog and discussion plugin are only done when the plugin is installed and no mail is accidentally lost when there are some keywords in the subject that triggers the blog or discussion functions.

[ibarton@…]

OK, I could do it too. Can I beg for commit tagged with "Blackhex" author once more, please?

[bas]

Waht do you mean with tagged? I can put in the Changelog that Blackhex is the author. All our trac users are in the central user database LDAP.

The next question you offer to do the regex stuff?

[ibarton@…]

I mean svn propset --revprop -r <revision> svn:author Blackhex for Ohloh statistics.

[ibarton@…]

I mean svn propset --revprop -r <revision> svn:author Blackhex for Ohloh statistics.

[bas]

Oke, but the project is not monitored by Ohloh.

[ibarton@…]

And can I add it?

[bas]

to Ohloh if you want it is fine for me. I have done some minor changes and gone add it to the trunk version shortly.

[bas]

(In [359]) email2trac.py.in:

• added DiscussionPlugin? support, Authors: bas en blackhex, closes #197

[bas]

Sorry, but have to create pre-revprop change hook or somethiing to change the author:

svn: DAV request failed; it's possible that the repository's pre-revprop-change hook either failed or is non-existent
svn: At least one property change failed; repository is unchanged
svn: Error setting property 'author': 
Repository has not been enabled to accept revision propchanges;
ask the administrator to create a pre-revprop-change hook

[bas]

Do you have a script that can do this?

[ibarton@…]

I recall that I have workarounded this by replacing default return value 1 of default pre-revrpop-change hook to 0:

REPOS="$1"
REV="$2"
USER="$3"
PROPNAME="$4"
ACTION="$5"

if [ "$ACTION" = "M" -a "$PROPNAME" = "svn:log" ]; then exit 0; fi

#echo "Changing revision properties other than svn:log is prohibited" >&2
#exit 1
exit 0

[bas]

oke, different error:

Wed May 19 16:02:12 2010] [error] [client 145.100.29.222] Could not execute PROPPATCH.  [500, #206]
[Wed May 19 16:02:12 2010] [error] [client 145.100.29.222] Revprop change blocked by pre-revprop-change hook (exit code 255) with no output.  [500, #165001]

[ibarton@…]

chmod a+x /path/to/repo/hooks/pre-revprop-change

?

[bas]

:-D