Modify

Opened 11 years ago

Closed 8 years ago

#25 closed task (fixed)

Question about pbs_python

Reported by: grva@… Owned by: bas
Priority: major Milestone:
Component: pbs Version: 4.3.0
Keywords: Cc:

Description

Hello,

I ran across your excellent pbs_python module. I'm having a bit of trouble
using it to submit jobs, and I was wondering if I could ask a very naive
question?

After installing, and checking that the pbsnodes-a.py works, I'm trying to
submit jobs by doing the following:

import pbs

# determine script_name, fill out attrop1, etc.

server_name = pbs.pbs_default()
c = pbs.pbs_connect(server_name)
job_id = pbs.pbs_submit(c,attrop1,script_name,'NULL','NULL')

e, e_txt = pbs.error()
if e:
     print e,e_txt
else:
     print "Submitted PBS Job " + str(job_id)

However, when I do this, I get "15020 Expired Credential in request", and I
don't know what this means, nor how to fix it.

It seems like I am missing something obvious, however, and I was hoping you
might be able to point it out.

Many thanks in advance.

Cheers,
Greg

---------------------------------------------------------------------------
Greg van Anders
Research Fellow
Glotzer Group, University of Michigan
---------------------------------------------------------------------------

Attachments (1)

smime.p7s (4.4 KB) - added by basv@… 11 years ago.
Added by email2trac

Download all attachments as: .zip

Change History (17)

comment:1 Changed 11 years ago by bas

  • Owner changed from somebody to bas
  • Status changed from new to assigned
  • Type changed from defect to task

Greg,

Can you submit jobs on the command-line via qsub. Maybe this host is not allowed to submit jobs. I am also missing the atrrop1 initialization. This example works for me:

import pbs

pbs_server = pbs.pbs_default ()
pbsconn = pbs.pbs_connect (pbs_server)

print pbsconn

attrl = pbs.new_attropl(1)
attrl[0].name = pbs.ATTR_N
attrl[0].value = "test"

task_id = pbs.pbs_submit(pbsconn, attrl, "A1.tsk", '', 'NULL')

e, e_txt = pbs.error()
if e:
        print e,e_txt

print task_id

comment:2 Changed 11 years ago by grva@…

Hello,

Many thanks for your reply!

I am able to submit this job using qsub, if I add what the contents of
attrop1 are to the top of the script with the normal #PBS stuff.

I added the following to the script

if ( verbose == 1 ):
     print "Found server:     " + server_name
     print "Connection value: " + str(c)
     for i in attrop1:
         if i.value:
             print " ".join([i.name,i.resource,i.value])
         else:
             print " ".join([i.name,i.resource])


and this is the result

Found server:     nyx.engin.umich.edu
Connection value: 1
Job_Name
/nobackup/grva/research/janus/chiral/run/script.py/hoomd_script.py_
Account_Name  glotzer
Join_Path  oe
destination  route
Resource_List nodes 1:tesla:glotzer
Resource_List pmem 1gb
Mail_Users grva@umich.edu
Mail_Points abe
Resource_List software graphics
Resource_List qos glotzergpu
Resource_List walltime 24:00:00

Some more information about my system:
$ qsub --version
version: 2.5.5
$ python --version
Python 2.6.4

Many thanks again, and if you need more information please let me know.

Cheers,
Greg

On Tue, 7 Jun 2011, pbs_python wrote:

> #25: Question about pbs_python
> ---------------------------+-----------------------
> Reporter:  grva@?          |       Owner:  bas
>    Type:  task            |      Status:  assigned
> Priority:  major           |   Component:  pbs
> Version:  4.3.0           |  Resolution:
> Keywords:                  |
> ---------------------------+-----------------------
> Changes (by bas):
>
> * owner:  somebody => bas
> * status:  new => assigned
> * type:  defect => task
>
>
> Comment:
>
> Greg,
>
>  Can you submit jobs on the command-line via qsub. Maybe this host is not
> allowed to submit jobs. I am also missing  the atrrop1 initialization.
> This example works for me:
> {{{
> import pbs
>
> pbs_server = pbs.pbs_default ()
> pbsconn = pbs.pbs_connect (pbs_server)
>
> print pbsconn
>
> attrl = pbs.new_attropl(1)
> attrl[0].name = pbs.ATTR_N
> attrl[0].value = "test"
>
> task_id = pbs.pbs_submit(pbsconn, attrl, "A1.tsk", '', 'NULL')
>
> e, e_txt = pbs.error()
> if e:
>         print e,e_txt
>
> print task_id
> }}}
>
> --
> Ticket URL: <https://subtrac.sara.nl/oss/pbs_python/ticket/25#comment:1>
> pbs_python <https://subtrac.sara.nl/oss/pbs_python>
> Documentation server for pbs_python
>
>
>

comment:3 Changed 11 years ago by anonymous

Did you try my simple example? Just to make sure you use the same user for qsub and your python script.

comment:4 Changed 11 years ago by grva@…

Ah! Sorry.

When I ran the example you suggested I got:

$ python test.py
1
15044 Resources temporarily unavailable
None

Thanks,
Greg

On Tue, 7 Jun 2011, pbs_python wrote:

> #25: Question about pbs_python
> ---------------------------+-----------------------
> Reporter:  grva@…          |       Owner:  bas
>    Type:  task            |      Status:  assigned
> Priority:  major           |   Component:  pbs
> Version:  4.3.0           |  Resolution:
> Keywords:                  |
> ---------------------------+-----------------------
> Changes (by bas):
>
> * owner:  somebody => bas
> * status:  new => assigned
> * type:  defect => task
>
>
> Comment:
>
> Greg,
>
>  Can you submit jobs on the command-line via qsub. Maybe this host is not
> allowed to submit jobs. I am also missing  the atrrop1 initialization.
> This example works for me:
> {{{
> import pbs
>
> pbs_server = pbs.pbs_default ()
> pbsconn = pbs.pbs_connect (pbs_server)
>
> print pbsconn
>
> attrl = pbs.new_attropl(1)
> attrl[0].name = pbs.ATTR_N
> attrl[0].value = "test"
>
> task_id = pbs.pbs_submit(pbsconn, attrl, "A1.tsk", '', 'NULL')
>
> e, e_txt = pbs.error()
> if e:
>         print e,e_txt
>
> print task_id
> }}}
>
> --
> Ticket URL: <https://subtrac.sara.nl/oss/pbs_python/ticket/25#comment:1>
> pbs_python <https://subtrac.sara.nl/oss/pbs_python>
> Documentation server for pbs_python
>
>
>

comment:5 Changed 11 years ago by bas

Which torque version do you use? and which pbs_python version? If find it strange that my simple example does not work. Do you have some kind of credentials system (kerberos) or a submitfilter implemented?

comment:6 Changed 11 years ago by grva@…

Hi,

My qsub says
$ qsub --version
version: 2.5.5

I assume this is the same version as torque. Is that right?

pbs_python version is 4.3.0 according to the tarball, but the python
.egg-info says 4.1.3.

We do use kerberos on campus, but not on the cluster, as far as I know. I
tried submitting both with and without active kerberos tickets and got the
same error message both ways.

I'm not sure if we have a submit filter, nor how to see if there is one. I
will check with the sysadmins about this.

Thanks,
Greg

On Thu, 9 Jun 2011, pbs_python wrote:

> #25: Question about pbs_python
> ---------------------------+-----------------------
> Reporter:  grva@?          |       Owner:  bas
>    Type:  task            |      Status:  assigned
> Priority:  major           |   Component:  pbs
> Version:  4.3.0           |  Resolution:
> Keywords:                  |
> ---------------------------+-----------------------
>
> Comment (by bas):
>
> Which torque version do you use? and which pbs_python version?  If find it
> strange that my simple example does not work. Do you have some kind of
> credentials system (kerberos) or a submitfilter implemented?
>
> --
> Ticket URL: <https://subtrac.sara.nl/oss/pbs_python/ticket/25#comment:5>
> pbs_python <https://subtrac.sara.nl/oss/pbs_python>
> Documentation server for pbs_python
>
>
>

comment:7 Changed 11 years ago by anonymous

It is still strange that the small example does not work. Further i tested the package only on 2.4.X installations. But i do not expect this to be a problem

comment:8 Changed 11 years ago by grva@…

Hi,

Sorry for the long delay in responding. A security issue took down our
cluster for some time and the admins were too busy to respond to my
request for info until now.

The version of torque is 2.5.5. There is no submit filter and the
authentication that happens upon submission is "pretty much the default".

I hope this information helps.

I'd be happy to try some other examples if you think they might help
figure out what's wrong.

Cheers,
Greg


On Fri, 10 Jun 2011, pbs_python wrote:

> #25: Question about pbs_python
> ---------------------------+-----------------------
> Reporter:  grva@?          |       Owner:  bas
>    Type:  task            |      Status:  assigned
> Priority:  major           |   Component:  pbs
> Version:  4.3.0           |  Resolution:
> Keywords:                  |
> ---------------------------+-----------------------
>
> Comment (by anonymous):
>
> It is still strange that the small example does not work. Further i tested
> the package only on 2.4.X installations. But i do not expect this to be a
> problem
>
> --
> Ticket URL: <https://subtrac.sara.nl/oss/pbs_python/ticket/25#comment:7>
> pbs_python <https://subtrac.sara.nl/oss/pbs_python>
> Documentation server for pbs_python
>
>
>

comment:9 Changed 11 years ago by bas

I completely forgot this issue. Sorry. I have to test the pbs_python package with torque 2.5 version. Currently we do not have this version installed. I will try to install it on and do some tests. Hopefully we can trace down the problem

comment:10 Changed 11 years ago by grva@…

Hi,

No problem. Our cluster was just down for maintenance and is now back up
(yesterday). qsub now reports the following version:

$ qsub --version
version: 2.5.7

I'm going to have a go at this and see if the software updates make a
difference.

Thanks!

On Thu, 28 Jul 2011, pbs_python wrote:

> #25: Question about pbs_python
> ---------------------------+-----------------------
> Reporter:  grva@…          |       Owner:  bas
>    Type:  task            |      Status:  assigned
> Priority:  major           |   Component:  pbs
> Version:  4.3.0           |  Resolution:
> Keywords:                  |
> ---------------------------+-----------------------
>
> Comment (by bas):
>
> I completely forgot this issue. Sorry. I have to test the pbs_python
> package with torque 2.5 version. Currently we do not have this version
> installed. I will try to install it on and do some tests. Hopefully we can
> trace down the problem
>
> --
> Ticket URL: <https://subtrac.sara.nl/oss/pbs_python/ticket/25#comment:9>
> pbs_python <https://subtrac.sara.nl/oss/pbs_python>
> Documentation server for pbs_python
>
>
>

comment:11 Changed 11 years ago by grva@…

Hello,

Ok now when I try your sample script from before

{{{
import pbs

pbs_server = pbs.pbs_default ()
pbsconn = pbs.pbs_connect (pbs_server)

print pbsconn

attrl = pbs.new_attropl(1)
attrl[0].name = pbs.ATTR_N
attrl[0].value = "test"

task_id = pbs.pbs_submit(pbsconn, attrl, "A1.tsk", '', 'NULL')

e, e_txt = pbs.error()
if e:
   print e,e_txt

print task_id
}}}

I get:
$ python test.py
1
15044 Resources temporarily unavailable
None

When I tried something with the normal -l, -A, -q flags I use when I
submit a job I get an error:
15041 Request to MOM failed

Does any of this help?

Thanks!

On Thu, 28 Jul 2011, pbs_python wrote:

> #25: Question about pbs_python
> ---------------------------+-----------------------
> Reporter:  grva@…          |       Owner:  bas
>    Type:  task            |      Status:  assigned
> Priority:  major           |   Component:  pbs
> Version:  4.3.0           |  Resolution:
> Keywords:                  |
> ---------------------------+-----------------------
>
> Comment (by grva@…):
>
> {{{
>
> Hi,
>
> No problem. Our cluster was just down for maintenance and is now back up
> (yesterday). qsub now reports the following version:
>
> $ qsub --version
> version: 2.5.7
>
> I'm going to have a go at this and see if the software updates make a
> difference.
>
> Thanks!
>
> On Thu, 28 Jul 2011, pbs_python wrote:
>
> > #25: Question about pbs_python
> > ---------------------------+-----------------------
> > Reporter:  grva@…          |       Owner:  bas
> >    Type:  task            |      Status:  assigned
> > Priority:  major           |   Component:  pbs
> > Version:  4.3.0           |  Resolution:
> > Keywords:                  |
> > ---------------------------+-----------------------
> >
> > Comment (by bas):
> >
> > I completely forgot this issue. Sorry. I have to test the pbs_python
> > package with torque 2.5 version. Currently we do not have this version
> > installed. I will try to install it on and do some tests. Hopefully we
> can
> > trace down the problem
> >
> > --
> > Ticket URL: <https://subtrac.sara.nl/oss/pbs_python/ticket/25#comment:9>
> > pbs_python <https://subtrac.sara.nl/oss/pbs_python>
> > Documentation server for pbs_python
> >
> >
> >
> }}}
>
> --
> Ticket URL: <https://subtrac.sara.nl/oss/pbs_python/ticket/25#comment:10>
> pbs_python <https://subtrac.sara.nl/oss/pbs_python>
> Documentation server for pbs_python
>
>
>

comment:12 follow-up: Changed 11 years ago by Glen Beane

I think this error may be misleading. I believe some of the values of the error constants may have changed in TORQUE 2.5 -- pbs_python includes a header file from a previous version of TORQUE that defines these. I got the same "expired credential" error when I had a typo in the queue name I was trying to submit to.

comment:13 in reply to: ↑ 12 Changed 11 years ago by bas

Replying to Glen Beane:

I think this error may be misleading. I believe some of the values of the error constants may have changed in TORQUE 2.5 -- pbs_python includes a header file from a previous version of TORQUE that defines these. I got the same "expired credential" error when I had a typo in the queue name I was trying to submit to.

It is on my todo list to port pbs_python to higer torque version then 2.4 (we use this version on our clusters). It think it will be 3.X series. The pbs_python is only test for the 2.4 and 2.1 versions. For torque version >= 2.4 it wil use the generate swig files for torque version 2.4. This means also the torque header file. So your are right if there are changes in the header file it will end up in the wrong error message.

comment:14 Changed 11 years ago by grva@…

> Comment (by bas):
>
> It is on my todo list to port pbs_python to higer torque version then 2.4
> (we use this version on our clusters). It think it will be 3.X series. The
> pbs_python is only test for the 2.4 and 2.1 versions.  For torque version
> >= 2.4 it wil use the generate swig files for torque version 2.4. This
> means also the torque header file. So your are right if there are changes
> in the header file it will end up in the wrong error message.
>

I didn't realize that pbs_python was swig based.

I have some experience with swig; would it be very difficult to port it to
torque v 2.5? I would just try it, but I don't see a swig module file in
the src directory.

comment:15 Changed 11 years ago by basv@…

On 08-11-11 15:08, Greg van Anders wrote:
>> Comment (by bas):
>>
>> It is on my todo list to port pbs_python to higer torque version then 2.4
>> (we use this version on our clusters). It think it will be 3.X series.
The
>> pbs_python is only test for the 2.4 and 2.1 versions.  For torque version
>>> = 2.4 it wil use the generate swig files for torque version 2.4. This
>> means also the torque header file. So your are right if there are changes
>> in the header file it will end up in the wrong error message.
>>
>
> I didn't realize that pbs_python was swig based.
>
> I have some experience with swig; would it be very difficult to port it to
> torque v 2.5? I would just try it, but I don't see a swig module file in
> the src directory.

;-) The swig file is not include because of swig version problems ( a lot
is changed between versions). The first pbs_python version it was included
, but i received a lot of emails that it could not compile the software. So
i decided to only to distribute the generated source files.

BTW) I have to make a automate the swig build process so that it parses the
pbs_ifl.h file and generate a pbs_ifl.h that is suitable for swig and
convert c-types to python-types (see pbs_ifl.h in pbs_python). It would
make life a lot easier. There are also some python functions added to the
swig generated files. To make it short it is not distribution ready.

smime.p7s

Changed 11 years ago by basv@…

Added by email2trac

comment:16 Changed 8 years ago by bas

  • Resolution set to fixed
  • Status changed from assigned to closed

the swig file is now added to devel branch

Add Comment

Modify Ticket

Change Properties
Action
as closed The owner will remain bas.
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.