source: trunk/CHANGES @ 279

Last change on this file since 279 was 279, checked in by bas, 12 years ago

fix a bug in getjob could not hamdle short jobname
  • Property svn:keywords set to Id
File size: 21.4 KB
Line 
1=========== 4.3.4
2 * PBSQuery bugfix. getjob() can now handle short jobname, eg:
3    * p.getjob('6073545')
4
5   Author: Willem Vermin (SARA) & Bas van der Vlies
6 
7=========== 4.3.3
8 * New generated files for pbs_wrap.c and pbs.py to support python 3.X versions
9   Reported by: Steve Traylen <steve dot traylen add cern dot ch>
10   Author: Bas van der Vlies
11
12 * Fixed AdvancedParser when using 01-12 range the zero was not appended.
13   Reported by: Ramon Bastiaans
14   Author: Dennis Stam
15
16 * examples/sara_nodes.py: Catch the PBSQuery error if we can not make a
17   connection with the batch server and exit the program.
18   Author: Bas van der Vlies
19
20 * Remove the debian dependency on AdvancedParser. We now have
21   our own PBSAdvanceParser so it does not conflict with other SARA
22   packages.
23   Reported by: Ramon Bastiaans (SARA)
24   Fixed by: Bas van der Vlies
25
26 * Make pbs_torque only dependend on libtorque instead of torque and adjusted
27   the maintainer of the package, closes #30
28   Reported by: Guillermo Marcus  <guillermo dot marcus add@gmail dot com>
29   Fixed by: Bas van der Vlies
30   
31=========== 4.3.0
32 * new_rack_pbsmon.py: For backward compatibility added -a/--all option.
33   Has the same functionality as -s/--summary
34   Author: Bas van der Vlies
35
36 * Added some license info to debian package and include GPL and LGPL 3.0
37   files.
38   Suggested by: W. Trevor King <wking at drexel dot edu>
39
40 * PBSQuery.py: Added support for new feature in node status field:
41    * message=EVENT:<key=value:>+
42    * message=ERROR <text>
43
44  Added the following syntax for node:
45        * node['error']
46    * node['event']
47
48  eg: node: n1
49    state = free
50    np = 8
51    properties = ib,switch1,highmem
52    ntype = cluster
53    jobs = 0/567403.sara.nl, 1/567403.sara.nl
54    status = ..,loadave=0.00,message=EVENT:sample.time=1288864220.003:cputotals.user=0:iconnect.pktout=0,netload=3487600394,state=free,...
55
56  example code:
57        p = PBSQuery()
58    node = p.getnode(n1)
59    print node.event['sample.time']
60    print node['event']['sample.time']
61
62   Requested by: Roy Dragseth <roy dot dragseth add uit dotno>
63   Author: Bas van der Vlies
64
65=========== 4.2.0
66 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
67   (jsbronder add gentoo dot org) for the information and
68   explanation.
69
70 * examples/pbsmon.py. Patched so it also works for torque 2.5
71   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
72   Applied by: Bas van der Vlies
73
74 * setup.py.in changed determine which torque version is installed
75   so we also support snapshots.
76   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
77   Applied by: Bas van der Vlies
78
79 * PBSQuery:
80   - get_nodes function for jobs now works for old and new datastructure and
81     gives correct results if unique=True
82     Reported by + idea for patch: Shunjie Lau (eijnuhs add gmail dot com)
83     Author: Bas van der Vlies
84
85 * Added AdvanceParser module. new_rack_pbsmon.py and sara_nodes.py make use of
86   this module
87   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
88
89 * examples/new_rack_pbsmon.py:
90     - Better specification for node expression
91     - added a new option -l/--list to list the state of a nodes and which job
92       is running on it, eg:
93             new_rack_pbsmon.py - l gb-r[1-3]n[3-6] gb-r4[n1-20]
94
95   Authors: Dennis Stam & Bas van der Vlies
96   Applied by: Bas van der Vlies
97
98 * examples/sara_nodes.py:
99   An utility to set/clear notes for multiple nodes, see ..
100   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
101
102 * setup.py.in:
103   Added support for runtime-libraries when torque is installed in a non-default
104   location, closes #22
105   Author: Mattias (slabanja at chalmers dot se)
106   Applied by: Bas van der Vlies
107
108 * Changed build system for debian
109   Authors: Dennis Stam & Bas van der Vlies
110
111=========== 4.1.2
112 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
113   structure.
114   Author: Bas van der Vlies
115
116 * PBSQuery:
117     - fixed and error in get_jobs function for nodes. Did not
118       support new data structure and uniq did not work.
119       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
120
121=========== 4.1.0
122 * Delete function pbs.version() replaced by
123    * pbs.version
124        This is a string variable
125    * pbs.version_info
126        Is a tuple variable. So we can use it in scripts
127
128   This analogue with sys module, closes #20
129
130   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
131   Author: Bas van der Vlies
132
133 * More robust make/configure/setup.py
134   Reported by:
135        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
136        - Shunjie Lau <eijnuhs add gmail dot com>
137   Author: Bas van der Vlies
138
139
140 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
141   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
142   Author: Bas van der Vlies
143   
144=========== 4.0.2
145  * Quick fix so we do not get any undefined symbols. Forgot to
146    add the torque library when we compile the package.
147    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
148    Author: Bas van der Vlies
149
150=========== 4.0.1
151  * Fixed a bug in setting the right library path  for compiling.
152    Reported by: Chad Vizino ( vizino add psc dot edu )
153    Author: Bas van der Vlies
154   
155=========== 4.0.0
156  * pbs.py added some new functions for 2.4 version:
157    - pbs_fbserver(void);
158    - pbs_get_server_list(void);
159    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
160    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
161    - pbs_checkpointjob(int connect, char *job_id, char *extend);
162    - log_ext(int,char *,char *,int);
163    - log_init(char *, char *);
164    - log_remove_old(char *,unsigned long);
165
166    Note: version 2.1.X is still supported
167
168  * autotools environment
169    - Added support for finding pbs-config (borrowed for mpiexec source)
170    - Detecting which version of torque is installed
171    - only support installations that have pbs-config installed.
172    - Removed old code for installations that do not a have pbs-config
173      installed.
174
175  * PBSQuery.py
176    - There was a bug in the new data interface (Resource_List for jobs)
177      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
178      Applied by with some minor changes: Bas van der Vlies
179
180    - The new data structure is the default. You can switch back to the old
181      one with:
182            p = PBSQuery()
183            p.old_data_structure()
184
185      Author: Bas van der Vlies
186
187   * examples/ha_server.py
188     An example how to use the High Availabilty torque functions
189     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
190
191=========== 3.6.0
192  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
193    Reported by: Dennis Stam
194    Fixed by: Bas van der Vlies
195
196=========== 3.5.0
197  * PBSQuery
198        The class functions of node, job and queue support old and new data
199        structure.
200
201        Changed the behaviour of the new data stucture, We can use it as
202        dictionary and as class attribute, this is equivalent, eg:
203          - print node['np'] and print node.np
204
205        for a node we parse the 'status' line and split on '=' char, You now can
206        use these statements, eg
207          - print node.status.arch     (node['status'].arch or node['status']['arch'])
208          - print node.status.nsession
209
210        for a job we parse the 'Variable_List' line and split on '=' char, You now can
211        use the statements, eg:
212          - print job.Variable_List.PBS_O_WORKDIR
213          - print job.Variable_List.PBS_O_HOME
214
215        For more info see examples/new_interface.py
216
217    Author: Bas van der Vlies
218
219 * new_rack_pbsmon.py
220        Rewrite to new data structure and automatically determine how many nodes
221        and racks cluster has and skip printing of empty racks (default), use -w/--wide
222        for old behaviour.
223
224   Author: Bas van der Vlies
225
226
227=========== 3.2.0
228        - PBSQuery:
229          New data structure. In the old structure it is a dictionary
230          with a value and the value is a string. This is changed
231          that dictionary values are now of type list or dictionary depends
232          on the value of the keyword, eg for a node:
233            - np = 2:
234              * node['np'] = [ '2' ] 
235
236            - properties = cores2, mem4gb, parallel
237              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
238
239            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
240              * node['status']['arch'] = [ 'x86_64' ]
241              * node['status']['sessions'] = [ '222599' ]
242              * ...
243
244         The data structure is activated by the function:
245          * new_data_structure()
246
247        In a future release it will be come the default.
248        example:
249                p = PBSQuery()
250                p.new_data_structure()
251
252                nodes = p.getnodes()
253                print nodes.np, nodes['np']
254
255        Author: Bas van der Vlies
256
257        - PBSQuery
258          For old and new data structure we now can use another syntax:
259           * node['np'] and node.np are equivalent
260
261          This new syntax works for all keywords.
262
263        Author: Bas van der Vlies
264
265        - Added iter object for job, node, queue and server objects, eg:
266          node = p.getnode('gb-r1n1')
267          print node.name
268          for attrib in node:
269                print '%\t%s = %s' %(attrib, node[attrib])
270        Author: Bas van der Vlies
271
272        - fixed an error in getnode, getqueue and getjob, return empty
273          dictionary if not found
274        Author: Bas van der Vlies
275       
276        - New build system for rpm packages, make -f Makefile.rpm
277          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
278
279        Applied: Bas van der Vlies
280
281=========== 3.0.1
282        - PBSQuery
283                * Removed a testing code line
284                Reported By: sam @ vpac dot org
285                Fixed by: Bas van der Vlies
286
287=========== 3.0.0
288        - PBSQuery
289                * Cleaned up some old lines of code which didn't do anything.
290                * Changed the returnvalue of getqueue, getnode and getjob. Now
291                  they return an instance of the object, instead of a dict.
292                  This is API change so we bumped the version to 3.0.0
293                * Now the getnode-function accepts the short name as well as
294                  the long name as an argument.
295
296                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
297
298        - pbs.py, _pbs.so, PBSQuery.py
299                * Fixed a memory leak when used in daemon mode
300                Author: Bas van der Vlies <basv@sara.nl>
301
302=========== 2.9.8
303        - pbs_python.spec file patch added libdir and python defines so it
304          will build for RHEL5.1/CentOS 5.1
305          by        : Michael Sternberg <sternberg add anl dot gov>
306          applied by: Bas van der Vlies
307
308        - Updated the header files for pbs_python. So it can use some new
309          defines variables/defines, eg:
310                * ATTR_NODE_status
311                * ATTR_NODE_note
312                * ...
313          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
314
315        - Two new functions to log.h:
316                log_roll(int size);
317                log_size(void);
318          Added by: Bas van der Vlies
319
320        - We now have proper destructors (free memory) in SWIG for attrl and
321          attropl attributes.
322          Added by: Bas van der Vlies
323
324        - PBSQuery
325                * Added getnodes_with_property() function. This will get all nodes with
326                  a certain property, eg:
327                   - get all nodes with property 'cores8' and return all attributes:
328                      getnodes_with_property('cores8')
329               
330                   - get all nodes with property 'cores2' and only return the
331                     attributes 'state' and 'properties':
332                      attrl = [ 'state', 'properties' ]
333                      getnodes_with_property('cores2', attrl)
334               
335                     Only for torque version > 2.1.0
336             
337                Added by: Bas van der Vlies
338
339                * Added to class node the function
340                 - get_jobs(self, unique=None)
341                   Returns a list of the currently running job-id('s) on the node.
342                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
343
344                * Added get_server_name() function.
345                  This will return the PBS-server's name.
346                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
347
348                * Added to class _PBSobject the function
349                  - uniq(self, list)
350                    Filters out unique items of a list.
351                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
352
353                * Added to class job the function
354                  - get_nodes(self, unique=None)
355                    Returns a list of the nodes which run the job.
356                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
357
358=========== Version 2.9.4
359        -  Function pbs.pbs_geterrmsg does not exits any more is
360           replaced by pbs.error():
361                errno, text = pbs.error()
362                print errno, text
363           Reported by: pk at q-leap dot com
364           Fixed by   : Bas van der Vlies
365
366        - PBSQuery.py
367           * Added get_version() for server class, returns the version of
368             the batch server
369             Added by    : Bas van der Vlies
370
371           * Added get_nodes() for job class, returns a list of nodes on
372             which the job is run
373             Added by    : Bas van der Vlies
374
375           * PBSQuery.py rewrite make use of UserDict module.
376             Suggested by: Ramon Bastiaans
377             Fixed by    : Bas van der Vlies
378
379=========== Version 2.9.2
380        - The function pbs_rescquery() did not work. Fixed it
381        - The function avail(), did not work fixed it
382        - The functions pbs_rescreserve and pbs_rescrelease did not work
383          fixed it
384
385        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
386          nodes was wrong and state down has topmost priority
387
388=========== Version 2.9.0
389        - Added support for new torque build system 2.1.0
390        - Added new_rack_pbsmon.py
391          Authors: Walter de Jong & Bas van der Vlies
392
393=========== Version 2.8.2
394
395Configure has a new option:
396        --with-pbsdir=PATH
397
398If set it will use PATH specified as argument to
399find the pbs/torque libraries. If unset it will use
400some useful defaults to find the libaries.
401Suggested by: Troy Baer
402
403PBSQuery.py:
404        - If we can not make a connection with the server
405          then raise an exception and let the programmer
406          decides what to do.
407
408pbs_python.spec:
409        - It can now handle mode bits
410          Author: Martin Pels from SARA
411       
412=========== Version 2.8.0
413
414setup.py:
415        - Added /usr/lib to PBS_LIB_DIR
416
417pbs.py:
418        - Added new function "pbs.error()". This function checks if
419          an error has occured with a pbs function, eg:
420            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
421            error_number, err_txt = pbs.error()
422            if error_number:
423                print error_number, err_txt
424
425          prints the following message if script A.tsk does not exists:
426            15042 (qsub) cannot access script file
427
428pbs_python.spec:
429        - to make rpm packages from the source package, initial version
430        Author: Martin Pels from SARA
431
432README:
433        - Now requires python 2.1 and higher
434        - Explain how to build DEBIAN package
435
436=========== Version 2.7.10
437
438PBSQuery.py:
439        Fixed an error. You always got all resources from eg nodes
440        even when you supplied an attribute list where you only
441        requested the 'state' of the node.
442
443        Updated the inline python documentation
444
445=========== Version 2.7.9
446
447Fixed an error the previous version was in DEBUG mode. So
448you get an lot undesired output on the screen.
449
450debian package now also generated a package with the write
451version info.
452
453=========== Version 2.7.8
454
455Setup.py:
456        Can also handle ROCKS installations + the version
457        info is now the same as pbs.py
458        Thanks to: Roy Dragseth
459
460=========== Version 2.7.7
461
462PBSQuery module:
463  It can now be used in daemon programs. The pbs_server closes
464  the connection after an certain amount of time. This is fixed,
465  before every query there is an new connection and if the query
466  is finished the connection is closed.
467 
468=========== Version 2.7.6
469
470Now setup.py can also handle old Oscar Installations.
471Thanks To: Robin Humble
472
473Fixed an print statement fix in new_interface.py:
474Thanks To: Robin Humble
475
476Made some improvements for PBSQuery module:
477        - Updated the documentation
478        - Added the has_key() function to all
479          PBSObjects. So that the behaviour is similar to
480          an dictionary
481          Suggested By: Ramon Bastiaans
482       
483=========== Version 2.7.5
484 Added PBSQuery module.  This module requires pbs.py and it a simple module
485 for querying the pbs server. The documentation is in the module. Use for
486 example ipython to read it.
487
488 Written By: Roy Dragseth
489             Bas van der Vlies
490
491 eg: ( see also examples/new_interface.py)
492
493 from PBSQuery import PBSQuery
494 p = PBSQuery()
495
496 jobs = p.getjobs()
497 for name, job in jobs.items()
498        print job
499
500=========== Version 2.7.4
501 New versions fixes an bug in function pbs.pbs_statnode().
502 Reported by: Keith Poirier
503
504 Configure support:
505 Contributed By: Yaroslav Halchenko
506
507 Debian package support:
508 Contributed By: Yaroslav Halchenko
509
510 Added Support for Debuging the interface, You must edit pbs_wrap.c
511 and search for SARA_DEBUG:
512        #define SARA_DEBUG 1
513 Implemented by: Ramon Bastiaans
514
515 examples/pbsmon.py:
516        - Fixed an error when regex fails to determine node number,
517          no status was displayed
518        - It will now display and 'j' if the node is free for the batch
519          system and a job runs on the node (SMP-systems).
520 Changed by: Bas van der Vlies
521
522 examples/rack_pbsmon.py:
523        - An pbsmon that display node info per rack
524        - Edit the rack_pbsmon.py to adjust the values. It requires
525          that hostname contains rack and node id's,
526          eg: gb-r<number>n<number>
527 Contributed by: Walter de Jong
528
529=========== Version 2.7.3
530 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
531 Resource and QUEing manager).
532
533 The interface support OPENPBS and TORQUE.
534
535=========== Version 2.7.2
536 Some minor changes to include files. Now the Scalable PBS keywords
537 are also supported ( server attribute names: node_ping_rate and
538 node_check_rate). This does not interfere with the openpbs software.
539
540 pbsmon.py could not handle an one node cluster. Fixed it.
541
542=========== Version 2.7
543 Forgot to wrap the pbs_statfree() function. So we could not free
544 allocated memory from functions that return 'struct batch_status *'
545 like pbs_statjob().
546
547 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
548 Daniel Olson
549
550=========== Version 2.6.1
551 Setup.py now checks if all openpbs libraries are installed to compile the
552 package.
553
554=========== Version 2.6
555
556 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
557 function. This bug prevented to pass 'struct attrl' variables to this
558 function. Thanks to Evelyn Shiu for reporting this bug.
559
560
561=========== Version 2.5
562 Added the OpenPbs logging functions (log.h and liblog.a). There
563 is an example in the examples directory: logpbs.py
564
565 Fixed a bug in examples/resmom_info.py used the default port for
566 pbs_resmon if getservbyname fails.
567
568 Fixed a bug in resmom code. If the user has no permissions to query
569 the pbs_mom daemon a empty string is returned. Now we check for
570 the empty string. So we do not get a python exception.
571
572 Fixed a bug in setup.py forgot the exit statement when we did not
573 find the pbs libraries.
574
575=========== Version 2.3
576 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
577
578 Removed some obsolete code.
579
580 As for now we make use of the distutils module. So the package
581 is automatically compiled and installed in proper directories, usage:
582    python setup.py install
583 This requires that the package 'distutils' must be installed. As for
584 version 1.6 and higher it is in the distribution.
585
586 Added a function that prints the pbs python interface version. So we
587 can check if we have the right interface version:
588 import pbs
589 print pbs.version()
590 
591
592=========== Version 2.2
593
594In this release we can query the pbs_mom daemon with the aid of
595resource management functions:
596   pbs.openrm()
597   pbs.closerm()
598   pbs.downrm()
599   pbs.configrm()
600   pbs.addreq()
601   pbs.allreq()
602   pbs.flushreq()
603   pbs.activereq()
604   pbs.fullresp()
605   pbs.getreq()
606
607There is also a python function:
608   pbs.get_mom_values(id, [list]):
609     id   - connection id of pbs.openrm(node-name, port-number)
610     list - Is a optional paramter. A list of resource keywords
611
612   If 'list' is not supplied then this function will get the values
613   for the standard resource keywords, eg uname, loadave, .. + 'arch'
614   depended keywords. The arch feature is only implemented for linux,
615   but it can easily be ported to other oses.
616
617   if 'list' is supplied then we only fetch values for the keywords
618   that are in the list.
619
620   The function returns a dictonary. The keys are the resource keywords.
621
622   See 'resmom_info.py' for a example of this new feature.
623
624=========== Version 2.0
625
626The previous version was a simple interface above the PBS C API LIB.  This
627interface has gone a major changes. The most noticeable change is  that
628the functions accept/return Python lists instead of C-lists.  So you can now
629use standard Python syntax for manipulating Python lists. If you are familar
630with the Python syntax then you will appreciate this new interface.
631
632NOTE:
633  This interface is NOT compatible with the old one
634
635Here are some guidelines to convert your code to the new one:
636
637The constructors for the struct has gone a major change:
638  old code:
639    temp = pbs.new_attrl();
640    attrl_p = pbs.attrlPtr(temp)
641    attrl_p.name = 'state'
642
643  new_code:
644     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
645     attr_l[0].name = 'state'
646
647
648The pbs_stat functions returns Python lists instead of C-lists. There is
649NO next field anymore:
650  old_code:
651    temp = pbs.new_attrl();
652    attrl_p = pbs.attrlPtr(temp)
653    attrl_p.this = 'NULL'
654
655    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
656
657    while nodes.this != 'NULL':
658      print nodes.name
659
660      node_attrl = nodes.attribs
661      while node_attrl.this != 'NULL':
662        print '\t', node_attrl.name, '=', node_attrl.value
663        node_attrl = node_attrl.next
664
665    nodes = nodes.next
666
667
668  new_code:
669    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
670    for node in nodes:
671      print node.name
672      for attrib in node.attribs:
673        print '\t', attrib.name, '=', attrib.value
674
675Another advantage is you can use the print statement to show the
676connect of attrl and attropl classes:
677   attr_l = pbs.new_attrl(2)
678   attr_l[0].name = 'bas'
679   attr_l[0].value = 'van der Vlies'
680
681   print  attr_l[0]
682   >> (bas,,van der Vlies)
683   
684I hope these examples illustrate the changes. If you specify a wrong type
685for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.