Modify

Opened 11 years ago

Closed 11 years ago

Last modified 11 years ago

#197 closed enhancement (fixed)

DiscussionPlugin support

Reported by: ibarton@… Owned by: bas
Priority: major Milestone: 1.4.0
Component: email2trac Version: 1.3.2
Keywords: Cc:

Description

Hello, email2trac developers.

I'm a developer of DiscussionPlugin for Trac [1] and one of its users
requested an e-mail posting support for it [2]. I see that a good way to
implement this would be to extend your email2trac set of scrips so I'm asking
if would it be acceptable for you to include to mainstream possible patch that
would add DiscussionPlugin support to email2trac or should I fork/reuse
email2trac (or something else) in standalone solution or start to implement
something from scratch?

I haven't studied email2trac code much for now so I don't know how to do it
exactly yet, but in case of possitive answer, I'd digg more into it. Anyway,
any hints or advices would be appreciated.

Thank you for making your stand on this topic.

[1] - https://trac-hacks.org/wiki/DiscussionPlugin
[2] - https://trac-hacks.org/ticket/6883

With best regards,

Attachments (2)

discussion_plugin_support.diff (12.7 KB) - added by bas 11 years ago.
Just added for reference
discussion_plugin_support2.diff (12.7 KB) - added by bas 11 years ago.
A new patch for adding support for discussion plugin

Download all attachments as: .zip

Change History (40)

comment:1 Changed 11 years ago by bas

  • Status changed from new to assigned
  • Type changed from defect to enhancement

We have already FullBlog? support in email2trac. So it is no problem to add DiscussionPlugin? support as well. The fullblog plugin is not actively maintained so mybe it will be removed again. I do not know al the ins and out of the Discussion plugin so if there are problems with it in email2trac can i send you an email for support?

PS) Just look how the FullBlog? support is added. This is a good starting point. email2trac can handle attachments. I do not know if your plugin supports it.

comment:2 Changed 11 years ago by ibarton@…

OK, thank you. I'll start with it at Monday. And yes, DiscussionPlugin? supports attachments.

comment:3 Changed 11 years ago by ibarton@…

I've done it but I can't add the diff as an attachment due to spam filter and it's too long to be pasted here. Any suggestions?

comment:4 Changed 11 years ago by ibarton@…

OK, I've putted it to my Dropbox: http://dl.dropbox.com/u/6882884/discussion_plugin_support.diff . If you have any comments or suggestions, just tell me and I'll change it. I didn't tested it on a "real" e-mail server just from the command line with saved messages. And one another wish, please: If you decide to commit it to the upstream, would it be possible to tag that commit with my nick "Blackhex", just for Ohloh statistics?

comment:5 Changed 11 years ago by geeksrik@…

[root@astrotrac email2trac-1.3.2]# patch < discussion_plugin_support-1.diff (Stripping trailing CRs from patch.) patching file email2trac.py.in Hunk #3 succeeded at 300 (offset -5 lines). Hunk #5 succeeded at 1065 (offset -65 lines). Hunk #7 succeeded at 1137 with fuzz 2 (offset -65 lines). Hunk #8 FAILED at 1368. Hunk #9 FAILED at 1475. Hunk #10 succeeded at 1843 (offset -8 lines). Hunk #11 succeeded at 1798 (offset -65 lines). Hunk #12 succeeded at 1881 (offset -8 lines). Hunk #13 succeeded at 1942 (offset -65 lines). Hunk #14 succeeded at 2100 (offset -8 lines). 2 out of 14 hunks FAILED -- saving rejects to file email2trac.py.in.rej

Changed 11 years ago by bas

Just added for reference

comment:6 Changed 11 years ago by bas

I have no problem at all with patching this againt the trunk version

comment:7 Changed 11 years ago by geeksrik@…

[root@astrotrac email2trac-1.3.2]# patch < discussion_plugin_support.diff patch unexpectedly ends in middle of line patch: Only garbage was found in the patch input.

comment:8 Changed 11 years ago by bas

I have a question about this code:

+               # TODO: Read server base URL from config.
+               # Create request object to mockup context creation.
+               #
+               environ = {'SERVER_PORT' : 80,
+                                  'SERVER_NAME' : 'test',
+                                  'REQUEST_METHOD' : 'POST',
+                                  'wsgi.url_scheme' : 'http',
+                                  'wsgi.input' : sys.stdin}
+               chrome =  {'links': {},
+                                  'scripts': [],
+                                  'ctxtnav': [],
+                                  'warnings': [],
+                                  'notices': []}
+               if self.env.base_url_for_redirect:
+                       environ['trac.base_url'] = self.env.base_url
+               self.req = Request(environ, None)
+               self.req.chrome = chrome
+               self.req.tz = 'missing'

Our trac installation only supports port 443 or does this not matter. Is this code only needed by the Discussion plugin or there also benefits for email2trac?

Is the Discussion plugin only available for trac 0.9:

 self.attach_attachments(context.content_parts, self.VERSION == 0.9)

It is a lot of code ;-). Thanks for the patch. Maybe i will use the Discussion plugin from one of my opensource projects.

comment:9 Changed 11 years ago by ibarton@…

geeksrik@…: If you do:

svn co https://subtrac.sara.nl/oss/svn/email2trac
cp discussion_plugin_support.diff email2trac/trunk
cd email2trac/trunk/discussion_plugin_support.diff
patch < d

it should work just fine.

bas: This kind of initialization is just to overcome somehow Trac internals initialization for some stuff with DiscussionPlugin?. I think that the actuall value is not used anywhere in email2trac or DiscussionPlugin? so you can keep it that way. But if I'm wrong, I'll change this to read server name and port from trac.ini.

Is the Discussion plugin only available for trac 0.9:

No, DiscussionPlugin? supports 0.9, 0.10, 0.11 and 0.12. I tested this patch just with 0.11 but it should work with 0.12 as well.

comment:10 Changed 11 years ago by anonymous

  • Milestone set to 1.4.0

I had a similar problem with implementing workflow for tickets. I ended up to fake a request:

req = Mock(authname='anonymous', perm=MockPerm(), args={}

Maybe we can do this also for The discussion plugin. So we can delete this code and move this to the function _create_context. This is the only place where self.req is used. Some pseudo code:

req = Mock(authname='anonymous', perm=MockPerm(), args={})
self.req.authname = self.author
self.req.perm = PermissionCache(self.env, self.author)
context = Context.from_request(self.req)

I only asked it, because you use:

 self.attach_attachments(context.content_parts, self.VERSION == 0.9)

It suppose to handle: True or False

comment:11 Changed 11 years ago by ibarton@…

I did it that way first, but it's not enough for DiscussionPlugin? since it uses SpamFilterPlugin? that needs request initialized more complexly.

comment:12 Changed 11 years ago by geeksrik@…

Ok patching worked now. But when I try running the utility this is what i get:

[root@astrotrac trunk]# /usr/local/bin/email2trac --project=trac < msg.txt

email2trac.py -- Email tickets to Trac.

A simple MTA filter to create Trac tickets from inbound emails.

Copyright 2005, Daniel Lundin <daniel@edgewall.com>
Copyright 2005, Edgewall Software

Authors:
  Bas van der Vlies <basv@sara.nl>
  Walter de Jong <walter@sara.nl>

The scripts reads emails from stdin and inserts directly into a Trac database.

How to use
----------
 * See https://subtrac.sara.nl/oss/email2trac/

 * Create an config file:
    [DEFAULT]                        # REQUIRED
    project      : /data/trac/test   # REQUIRED
    debug        : 1                 # OPTIONAL, if set print some DEBUG info
    trac_version : 0.10              # OPTIONAL, default is 0.11

    [jouvin]                         # OPTIONAL project declaration, if set both fields necessary
    project      : /data/trac/jouvin # use -p|--project jouvin.

 * default config file is : /etc/email2trac.conf

 * Commandline opions:
                -h,--help
                -f,--file  <configuration file>
                -n,--dry-run
                -p, --project <project name>
                -t, --ticket_prefix <name>

SVN Info:
        $Id: email2trac.py.in 356 2010-04-15 13:18:09Z bas $

   No Trac project is defined in the email2trac config file.

this is my conf file in the same folder:

[DEFAULT]
project: /trac-repo
debug: 0
umask: 022
spam_level: 5
reply_all : 0
mailto_link: 0
umask: 022
email_header: 0
trac_version: 0.11
enable_syslog : 1
alternate_notify_template :
alternate_notify_template_update :
drop_spam : 0
verbatim_format: 1
strip_signature: 0
email_quote: >
strip_quotes: 0
ignore_trac_user_settings: 0
black_list: MAILER-DAEMON@
drop_alternative_html_version: 1

[tracproj]
project: /trac-repo
debug: 0
umask: 022
spam_level: 5
reply_all : 0
mailto_link: 0
umask: 022
email_header: 0
trac_version: 0.11
enable_syslog : 1
alternate_notify_template :
alternate_notify_template_update :
drop_spam : 0
verbatim_format: 1
strip_signature: 0
email_quote: >
strip_quotes: 0
ignore_trac_user_settings: 0
black_list: MAILER-DAEMON@
drop_alternative_html_version: 1

[bas]
project: /data/trac/bas
spam_level: 1

comment:13 Changed 11 years ago by geeksrik@…

Blackhex, i am adding this query in this ticket itself.

I have a mailbox which accepts emails. Lets say this is "mybox@…", now i want trac discussion plugin to be able to send mails (notificiations) to trac users, and the reply id must be "mybox@…". Assuming someone replies to a discussion, this mail will be left in "mybox@…". A cron job which is running then needs to read this mailbox, route that email to discussion plugin which will put the content at its appropriate place.

Now having said this, I am trying to understand what email2trac does. Firstly I believe this plugin does not scan mailboxes like "mybox@…" (pop or imap). secondly does your diff (patch) actually do this scanning of mailbox? what is the exact cron job i need to run in order that above process happens every minute or two? currently, i am only still getting notifications from DP, but any replies after setting above reply to address does not get picked up by DP.

Can the email2trac plugin provide POP mailbox scanning?

Any help in detailing this a bit will be much appreciated. thanks!

comment:14 Changed 11 years ago by anonymous

You don't have properly edited your email2trac.conf. Mine looks like this (when I'm running the script with --project=Test-0.11):

[Test-0.11]
project: /var/lib/trac/Test-0.11
debug: 5
umask: 022
spam_level: 5
reply_all : 0
mailto_link: 0
umask: 022
email_header: 0
trac_version: 0.11
enable_syslog : 0
alternate_notify_template :
alternate_notify_template_update :
drop_spam : 0
verbatim_format: 1
strip_signature: 0
email_quote: >
strip_quotes: 0
ignore_trac_user_settings: 0
black_list: MAILER-DAEMON@
drop_alternative_html_version: 1

comment:15 Changed 11 years ago by geeksrik@…

I dont think mine looks any different, but this is what i got:

[root@astrotrac trunk]# /usr/local/bin/email2trac --project=tracproj < msg.txt
Not a valid project name: tracproj
Valid names: [u'bas', 'trac']
[root@astrotrac trunk]#

this is my new config file:

[tracproj]
project: /trac-repo
debug: 0
umask: 022
spam_level: 5
reply_all : 0
mailto_link: 0
umask: 022
email_header: 0
trac_version: 0.11
enable_syslog : 1
alternate_notify_template :
alternate_notify_template_update :
drop_spam : 0
verbatim_format: 1
strip_signature: 0
email_quote: >
strip_quotes: 0
ignore_trac_user_settings: 0
black_list: MAILER-DAEMON@
drop_alternative_html_version: 1

comment:16 Changed 11 years ago by geeksrik@…

ok, i figured that out! it was to copied to /usr/local/etc instead of /usr/etc

[root@astrotrac trunk]# locate email2trac.conf
/opt/email2trac/email2trac-1.2.0/email2trac.conf
/opt/email2trac-1.2.0/email2trac.conf
/usr/local/etc/email2trac.conf
[root@astrotrac trunk]# cp email2trac.conf /usr/local/etc/.
cp: overwrite `/usr/local/etc/./email2trac.conf'? y
[root@astrotrac trunk]# /usr/local/bin/email2trac --project=tracproj < msg.txt
Loading environment /trac-repo
[root@astrotrac trunk]#

comment:17 Changed 11 years ago by geeksrik@…

Now for the old question: Can the email2trac plugin provide POP mailbox scanning?

comment:18 Changed 11 years ago by anonymous

I'm not its developer but as I know it can't. It just imports the file with e-mail message. You'd have to write a simple script that downloads the messages and then executes email2trac on them.

comment:19 Changed 11 years ago by bas

Just read this:

And read the Fetchmail setup, maybe this is what you want!

comment:20 Changed 11 years ago by bas

There are several discussions in this ticket ;-). But for the SpamFilterPlugin you need this complex setup. email2trac also handles spam it relies on programs that set t a flag in the email header. Maybe we can create a fake request that also init the SpamFilterPlugin. I will do some testing.

comment:21 Changed 11 years ago by bas

I have done some little rewriting and moved the statements that where in the class to the function:

  • _create_context

So it is easy for me what is mime en what belongs to the discussion plugin and the import statements are moved to the appropriate sectionm see new attachment. Removed also the print statement to stdout that shows which environment was loaded. Most MTA do not like that

Changed 11 years ago by bas

A new patch for adding support for discussion plugin

comment:22 Changed 11 years ago by ibarton@…

I'll check if that doesn't break anyting later...

comment:23 Changed 11 years ago by ibarton@…

OK, it didn't.

comment:24 Changed 11 years ago by bas

Thanks for testing. Was a bit swamped with a new project for our cluster installation tool. I will apply it asap to the trunk version and then make a new release. I will do some fine tuning that the regex for blog and discussion plugin are only done when the plugin is installed and no mail is accidentally lost when there are some keywords in the subject that triggers the blog or discussion functions.

comment:25 Changed 11 years ago by ibarton@…

OK, I could do it too. Can I beg for commit tagged with "Blackhex" author once more, please?

comment:26 Changed 11 years ago by bas

Waht do you mean with tagged? I can put in the Changelog that Blackhex is the author. All our trac users are in the central user database LDAP.

The next question you offer to do the regex stuff?

comment:27 Changed 11 years ago by ibarton@…

I mean svn propset --revprop -r <revision> svn:author Blackhex for Ohloh statistics.

comment:27 Changed 11 years ago by ibarton@…

I mean svn propset --revprop -r <revision> svn:author Blackhex for Ohloh statistics.

comment:28 Changed 11 years ago by bas

Oke, but the project is not monitored by Ohloh.

comment:29 Changed 11 years ago by ibarton@…

And can I add it?

comment:30 Changed 11 years ago by bas

to Ohloh if you want it is fine for me. I have done some minor changes and gone add it to the trunk version shortly.

comment:32 Changed 11 years ago by bas

  • Status changed from assigned to closed

(In [359]) email2trac.py.in:

comment:33 Changed 11 years ago by bas

Sorry, but have to create pre-revprop change hook or somethiing to change the author:

svn: DAV request failed; it's possible that the repository's pre-revprop-change hook either failed or is non-existent
svn: At least one property change failed; repository is unchanged
svn: Error setting property 'author': 
Repository has not been enabled to accept revision propchanges;
ask the administrator to create a pre-revprop-change hook

comment:34 Changed 11 years ago by bas

Do you have a script that can do this?

comment:35 Changed 11 years ago by ibarton@…

I recall that I have workarounded this by replacing default return value 1 of default pre-revrpop-change hook to 0:

REPOS="$1"
REV="$2"
USER="$3"
PROPNAME="$4"
ACTION="$5"

if [ "$ACTION" = "M" -a "$PROPNAME" = "svn:log" ]; then exit 0; fi

#echo "Changing revision properties other than svn:log is prohibited" >&2
#exit 1
exit 0

comment:36 Changed 11 years ago by bas

oke, different error:

Wed May 19 16:02:12 2010] [error] [client 145.100.29.222] Could not execute PROPPATCH.  [500, #206]
[Wed May 19 16:02:12 2010] [error] [client 145.100.29.222] Revprop change blocked by pre-revprop-change hook (exit code 255) with no output.  [500, #165001]

comment:37 Changed 11 years ago by ibarton@…

chmod a+x /path/to/repo/hooks/pre-revprop-change

?

comment:38 Changed 11 years ago by bas

:-D

Add Comment

Modify Ticket

Change Properties
Action
as closed The owner will remain bas.
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.