source: trunk/CHANGES @ 291

Last change on this file since 291 was 291, checked in by bas, 11 years ago

preparing for release 4.3.5
  • Property svn:keywords set to Id
File size: 22.0 KB
Line 
1=========== 4.3.5
2 * PBSQuery bugfix, parsing of node status line with multiple EVENT: settings
3   was wrong
4   Author: Bas van der Vlies
5
6 * Added an example how to make use of the EVENT status message:
7    - examples/pbs_script.sh
8   Author: Bas van der Vlies
9
10 * implemented __nonzero__ for PBS dictionary, closes #33, eg:
11    status = getjob(id)
12    if status:
13        do something
14    else:
15        no job
16  Author: Bas van der Vlies
17
18 * PBSQuery bugfix. getjob() can now handle short jobname, eg:
19    * p.getjob('6073545')
20
21   Author: Willem Vermin (SARA) & Bas van der Vlies
22
23 * added an example: pbs_jobmonitor shows the output of top
24   -u user on the node
25    - one cycle of top
26    - user: the user the job belongs to
27
28   renamed as pbs_joblogin logs in to the node as the user
29   who invokes this script (os.getenv('USER'))
30
31   Author: Willem Vermin (SARA)
32 
33=========== 4.3.3
34 * New generated files for pbs_wrap.c and pbs.py to support python 3.X versions
35   Reported by: Steve Traylen <steve dot traylen add cern dot ch>
36   Author: Bas van der Vlies
37
38 * Fixed AdvancedParser when using 01-12 range the zero was not appended.
39   Reported by: Ramon Bastiaans
40   Author: Dennis Stam
41
42 * examples/sara_nodes.py: Catch the PBSQuery error if we can not make a
43   connection with the batch server and exit the program.
44   Author: Bas van der Vlies
45
46 * Remove the debian dependency on AdvancedParser. We now have
47   our own PBSAdvanceParser so it does not conflict with other SARA
48   packages.
49   Reported by: Ramon Bastiaans (SARA)
50   Fixed by: Bas van der Vlies
51
52 * Make pbs_torque only dependend on libtorque instead of torque and adjusted
53   the maintainer of the package, closes #30
54   Reported by: Guillermo Marcus  <guillermo dot marcus add@gmail dot com>
55   Fixed by: Bas van der Vlies
56   
57=========== 4.3.0
58 * new_rack_pbsmon.py: For backward compatibility added -a/--all option.
59   Has the same functionality as -s/--summary
60   Author: Bas van der Vlies
61
62 * Added some license info to debian package and include GPL and LGPL 3.0
63   files.
64   Suggested by: W. Trevor King <wking at drexel dot edu>
65
66 * PBSQuery.py: Added support for new feature in node status field:
67    * message=EVENT:<key=value:>+
68    * message=ERROR <text>
69
70  Added the following syntax for node:
71        * node['error']
72    * node['event']
73
74  eg: node: n1
75    state = free
76    np = 8
77    properties = ib,switch1,highmem
78    ntype = cluster
79    jobs = 0/567403.sara.nl, 1/567403.sara.nl
80    status = ..,loadave=0.00,message=EVENT:sample.time=1288864220.003:cputotals.user=0:iconnect.pktout=0,netload=3487600394,state=free,...
81
82  example code:
83        p = PBSQuery()
84    node = p.getnode(n1)
85    print node.event['sample.time']
86    print node['event']['sample.time']
87
88   Requested by: Roy Dragseth <roy dot dragseth add uit dotno>
89   Author: Bas van der Vlies
90
91=========== 4.2.0
92 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
93   (jsbronder add gentoo dot org) for the information and
94   explanation.
95
96 * examples/pbsmon.py. Patched so it also works for torque 2.5
97   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
98   Applied by: Bas van der Vlies
99
100 * setup.py.in changed determine which torque version is installed
101   so we also support snapshots.
102   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
103   Applied by: Bas van der Vlies
104
105 * PBSQuery:
106   - get_nodes function for jobs now works for old and new datastructure and
107     gives correct results if unique=True
108     Reported by + idea for patch: Shunjie Lau (eijnuhs add gmail dot com)
109     Author: Bas van der Vlies
110
111 * Added AdvanceParser module. new_rack_pbsmon.py and sara_nodes.py make use of
112   this module
113   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
114
115 * examples/new_rack_pbsmon.py:
116     - Better specification for node expression
117     - added a new option -l/--list to list the state of a nodes and which job
118       is running on it, eg:
119             new_rack_pbsmon.py - l gb-r[1-3]n[3-6] gb-r4[n1-20]
120
121   Authors: Dennis Stam & Bas van der Vlies
122   Applied by: Bas van der Vlies
123
124 * examples/sara_nodes.py:
125   An utility to set/clear notes for multiple nodes, see ..
126   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
127
128 * setup.py.in:
129   Added support for runtime-libraries when torque is installed in a non-default
130   location, closes #22
131   Author: Mattias (slabanja at chalmers dot se)
132   Applied by: Bas van der Vlies
133
134 * Changed build system for debian
135   Authors: Dennis Stam & Bas van der Vlies
136
137=========== 4.1.2
138 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
139   structure.
140   Author: Bas van der Vlies
141
142 * PBSQuery:
143     - fixed and error in get_jobs function for nodes. Did not
144       support new data structure and uniq did not work.
145       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
146
147=========== 4.1.0
148 * Delete function pbs.version() replaced by
149    * pbs.version
150        This is a string variable
151    * pbs.version_info
152        Is a tuple variable. So we can use it in scripts
153
154   This analogue with sys module, closes #20
155
156   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
157   Author: Bas van der Vlies
158
159 * More robust make/configure/setup.py
160   Reported by:
161        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
162        - Shunjie Lau <eijnuhs add gmail dot com>
163   Author: Bas van der Vlies
164
165
166 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
167   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
168   Author: Bas van der Vlies
169   
170=========== 4.0.2
171  * Quick fix so we do not get any undefined symbols. Forgot to
172    add the torque library when we compile the package.
173    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
174    Author: Bas van der Vlies
175
176=========== 4.0.1
177  * Fixed a bug in setting the right library path  for compiling.
178    Reported by: Chad Vizino ( vizino add psc dot edu )
179    Author: Bas van der Vlies
180   
181=========== 4.0.0
182  * pbs.py added some new functions for 2.4 version:
183    - pbs_fbserver(void);
184    - pbs_get_server_list(void);
185    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
186    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
187    - pbs_checkpointjob(int connect, char *job_id, char *extend);
188    - log_ext(int,char *,char *,int);
189    - log_init(char *, char *);
190    - log_remove_old(char *,unsigned long);
191
192    Note: version 2.1.X is still supported
193
194  * autotools environment
195    - Added support for finding pbs-config (borrowed for mpiexec source)
196    - Detecting which version of torque is installed
197    - only support installations that have pbs-config installed.
198    - Removed old code for installations that do not a have pbs-config
199      installed.
200
201  * PBSQuery.py
202    - There was a bug in the new data interface (Resource_List for jobs)
203      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
204      Applied by with some minor changes: Bas van der Vlies
205
206    - The new data structure is the default. You can switch back to the old
207      one with:
208            p = PBSQuery()
209            p.old_data_structure()
210
211      Author: Bas van der Vlies
212
213   * examples/ha_server.py
214     An example how to use the High Availabilty torque functions
215     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
216
217=========== 3.6.0
218  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
219    Reported by: Dennis Stam
220    Fixed by: Bas van der Vlies
221
222=========== 3.5.0
223  * PBSQuery
224        The class functions of node, job and queue support old and new data
225        structure.
226
227        Changed the behaviour of the new data stucture, We can use it as
228        dictionary and as class attribute, this is equivalent, eg:
229          - print node['np'] and print node.np
230
231        for a node we parse the 'status' line and split on '=' char, You now can
232        use these statements, eg
233          - print node.status.arch     (node['status'].arch or node['status']['arch'])
234          - print node.status.nsession
235
236        for a job we parse the 'Variable_List' line and split on '=' char, You now can
237        use the statements, eg:
238          - print job.Variable_List.PBS_O_WORKDIR
239          - print job.Variable_List.PBS_O_HOME
240
241        For more info see examples/new_interface.py
242
243    Author: Bas van der Vlies
244
245 * new_rack_pbsmon.py
246        Rewrite to new data structure and automatically determine how many nodes
247        and racks cluster has and skip printing of empty racks (default), use -w/--wide
248        for old behaviour.
249
250   Author: Bas van der Vlies
251
252
253=========== 3.2.0
254        - PBSQuery:
255          New data structure. In the old structure it is a dictionary
256          with a value and the value is a string. This is changed
257          that dictionary values are now of type list or dictionary depends
258          on the value of the keyword, eg for a node:
259            - np = 2:
260              * node['np'] = [ '2' ] 
261
262            - properties = cores2, mem4gb, parallel
263              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
264
265            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
266              * node['status']['arch'] = [ 'x86_64' ]
267              * node['status']['sessions'] = [ '222599' ]
268              * ...
269
270         The data structure is activated by the function:
271          * new_data_structure()
272
273        In a future release it will be come the default.
274        example:
275                p = PBSQuery()
276                p.new_data_structure()
277
278                nodes = p.getnodes()
279                print nodes.np, nodes['np']
280
281        Author: Bas van der Vlies
282
283        - PBSQuery
284          For old and new data structure we now can use another syntax:
285           * node['np'] and node.np are equivalent
286
287          This new syntax works for all keywords.
288
289        Author: Bas van der Vlies
290
291        - Added iter object for job, node, queue and server objects, eg:
292          node = p.getnode('gb-r1n1')
293          print node.name
294          for attrib in node:
295                print '%\t%s = %s' %(attrib, node[attrib])
296        Author: Bas van der Vlies
297
298        - fixed an error in getnode, getqueue and getjob, return empty
299          dictionary if not found
300        Author: Bas van der Vlies
301       
302        - New build system for rpm packages, make -f Makefile.rpm
303          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
304
305        Applied: Bas van der Vlies
306
307=========== 3.0.1
308        - PBSQuery
309                * Removed a testing code line
310                Reported By: sam @ vpac dot org
311                Fixed by: Bas van der Vlies
312
313=========== 3.0.0
314        - PBSQuery
315                * Cleaned up some old lines of code which didn't do anything.
316                * Changed the returnvalue of getqueue, getnode and getjob. Now
317                  they return an instance of the object, instead of a dict.
318                  This is API change so we bumped the version to 3.0.0
319                * Now the getnode-function accepts the short name as well as
320                  the long name as an argument.
321
322                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
323
324        - pbs.py, _pbs.so, PBSQuery.py
325                * Fixed a memory leak when used in daemon mode
326                Author: Bas van der Vlies <basv@sara.nl>
327
328=========== 2.9.8
329        - pbs_python.spec file patch added libdir and python defines so it
330          will build for RHEL5.1/CentOS 5.1
331          by        : Michael Sternberg <sternberg add anl dot gov>
332          applied by: Bas van der Vlies
333
334        - Updated the header files for pbs_python. So it can use some new
335          defines variables/defines, eg:
336                * ATTR_NODE_status
337                * ATTR_NODE_note
338                * ...
339          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
340
341        - Two new functions to log.h:
342                log_roll(int size);
343                log_size(void);
344          Added by: Bas van der Vlies
345
346        - We now have proper destructors (free memory) in SWIG for attrl and
347          attropl attributes.
348          Added by: Bas van der Vlies
349
350        - PBSQuery
351                * Added getnodes_with_property() function. This will get all nodes with
352                  a certain property, eg:
353                   - get all nodes with property 'cores8' and return all attributes:
354                      getnodes_with_property('cores8')
355               
356                   - get all nodes with property 'cores2' and only return the
357                     attributes 'state' and 'properties':
358                      attrl = [ 'state', 'properties' ]
359                      getnodes_with_property('cores2', attrl)
360               
361                     Only for torque version > 2.1.0
362             
363                Added by: Bas van der Vlies
364
365                * Added to class node the function
366                 - get_jobs(self, unique=None)
367                   Returns a list of the currently running job-id('s) on the node.
368                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
369
370                * Added get_server_name() function.
371                  This will return the PBS-server's name.
372                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
373
374                * Added to class _PBSobject the function
375                  - uniq(self, list)
376                    Filters out unique items of a list.
377                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
378
379                * Added to class job the function
380                  - get_nodes(self, unique=None)
381                    Returns a list of the nodes which run the job.
382                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
383
384=========== Version 2.9.4
385        -  Function pbs.pbs_geterrmsg does not exits any more is
386           replaced by pbs.error():
387                errno, text = pbs.error()
388                print errno, text
389           Reported by: pk at q-leap dot com
390           Fixed by   : Bas van der Vlies
391
392        - PBSQuery.py
393           * Added get_version() for server class, returns the version of
394             the batch server
395             Added by    : Bas van der Vlies
396
397           * Added get_nodes() for job class, returns a list of nodes on
398             which the job is run
399             Added by    : Bas van der Vlies
400
401           * PBSQuery.py rewrite make use of UserDict module.
402             Suggested by: Ramon Bastiaans
403             Fixed by    : Bas van der Vlies
404
405=========== Version 2.9.2
406        - The function pbs_rescquery() did not work. Fixed it
407        - The function avail(), did not work fixed it
408        - The functions pbs_rescreserve and pbs_rescrelease did not work
409          fixed it
410
411        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
412          nodes was wrong and state down has topmost priority
413
414=========== Version 2.9.0
415        - Added support for new torque build system 2.1.0
416        - Added new_rack_pbsmon.py
417          Authors: Walter de Jong & Bas van der Vlies
418
419=========== Version 2.8.2
420
421Configure has a new option:
422        --with-pbsdir=PATH
423
424If set it will use PATH specified as argument to
425find the pbs/torque libraries. If unset it will use
426some useful defaults to find the libaries.
427Suggested by: Troy Baer
428
429PBSQuery.py:
430        - If we can not make a connection with the server
431          then raise an exception and let the programmer
432          decides what to do.
433
434pbs_python.spec:
435        - It can now handle mode bits
436          Author: Martin Pels from SARA
437       
438=========== Version 2.8.0
439
440setup.py:
441        - Added /usr/lib to PBS_LIB_DIR
442
443pbs.py:
444        - Added new function "pbs.error()". This function checks if
445          an error has occured with a pbs function, eg:
446            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
447            error_number, err_txt = pbs.error()
448            if error_number:
449                print error_number, err_txt
450
451          prints the following message if script A.tsk does not exists:
452            15042 (qsub) cannot access script file
453
454pbs_python.spec:
455        - to make rpm packages from the source package, initial version
456        Author: Martin Pels from SARA
457
458README:
459        - Now requires python 2.1 and higher
460        - Explain how to build DEBIAN package
461
462=========== Version 2.7.10
463
464PBSQuery.py:
465        Fixed an error. You always got all resources from eg nodes
466        even when you supplied an attribute list where you only
467        requested the 'state' of the node.
468
469        Updated the inline python documentation
470
471=========== Version 2.7.9
472
473Fixed an error the previous version was in DEBUG mode. So
474you get an lot undesired output on the screen.
475
476debian package now also generated a package with the write
477version info.
478
479=========== Version 2.7.8
480
481Setup.py:
482        Can also handle ROCKS installations + the version
483        info is now the same as pbs.py
484        Thanks to: Roy Dragseth
485
486=========== Version 2.7.7
487
488PBSQuery module:
489  It can now be used in daemon programs. The pbs_server closes
490  the connection after an certain amount of time. This is fixed,
491  before every query there is an new connection and if the query
492  is finished the connection is closed.
493 
494=========== Version 2.7.6
495
496Now setup.py can also handle old Oscar Installations.
497Thanks To: Robin Humble
498
499Fixed an print statement fix in new_interface.py:
500Thanks To: Robin Humble
501
502Made some improvements for PBSQuery module:
503        - Updated the documentation
504        - Added the has_key() function to all
505          PBSObjects. So that the behaviour is similar to
506          an dictionary
507          Suggested By: Ramon Bastiaans
508       
509=========== Version 2.7.5
510 Added PBSQuery module.  This module requires pbs.py and it a simple module
511 for querying the pbs server. The documentation is in the module. Use for
512 example ipython to read it.
513
514 Written By: Roy Dragseth
515             Bas van der Vlies
516
517 eg: ( see also examples/new_interface.py)
518
519 from PBSQuery import PBSQuery
520 p = PBSQuery()
521
522 jobs = p.getjobs()
523 for name, job in jobs.items()
524        print job
525
526=========== Version 2.7.4
527 New versions fixes an bug in function pbs.pbs_statnode().
528 Reported by: Keith Poirier
529
530 Configure support:
531 Contributed By: Yaroslav Halchenko
532
533 Debian package support:
534 Contributed By: Yaroslav Halchenko
535
536 Added Support for Debuging the interface, You must edit pbs_wrap.c
537 and search for SARA_DEBUG:
538        #define SARA_DEBUG 1
539 Implemented by: Ramon Bastiaans
540
541 examples/pbsmon.py:
542        - Fixed an error when regex fails to determine node number,
543          no status was displayed
544        - It will now display and 'j' if the node is free for the batch
545          system and a job runs on the node (SMP-systems).
546 Changed by: Bas van der Vlies
547
548 examples/rack_pbsmon.py:
549        - An pbsmon that display node info per rack
550        - Edit the rack_pbsmon.py to adjust the values. It requires
551          that hostname contains rack and node id's,
552          eg: gb-r<number>n<number>
553 Contributed by: Walter de Jong
554
555=========== Version 2.7.3
556 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
557 Resource and QUEing manager).
558
559 The interface support OPENPBS and TORQUE.
560
561=========== Version 2.7.2
562 Some minor changes to include files. Now the Scalable PBS keywords
563 are also supported ( server attribute names: node_ping_rate and
564 node_check_rate). This does not interfere with the openpbs software.
565
566 pbsmon.py could not handle an one node cluster. Fixed it.
567
568=========== Version 2.7
569 Forgot to wrap the pbs_statfree() function. So we could not free
570 allocated memory from functions that return 'struct batch_status *'
571 like pbs_statjob().
572
573 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
574 Daniel Olson
575
576=========== Version 2.6.1
577 Setup.py now checks if all openpbs libraries are installed to compile the
578 package.
579
580=========== Version 2.6
581
582 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
583 function. This bug prevented to pass 'struct attrl' variables to this
584 function. Thanks to Evelyn Shiu for reporting this bug.
585
586
587=========== Version 2.5
588 Added the OpenPbs logging functions (log.h and liblog.a). There
589 is an example in the examples directory: logpbs.py
590
591 Fixed a bug in examples/resmom_info.py used the default port for
592 pbs_resmon if getservbyname fails.
593
594 Fixed a bug in resmom code. If the user has no permissions to query
595 the pbs_mom daemon a empty string is returned. Now we check for
596 the empty string. So we do not get a python exception.
597
598 Fixed a bug in setup.py forgot the exit statement when we did not
599 find the pbs libraries.
600
601=========== Version 2.3
602 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
603
604 Removed some obsolete code.
605
606 As for now we make use of the distutils module. So the package
607 is automatically compiled and installed in proper directories, usage:
608    python setup.py install
609 This requires that the package 'distutils' must be installed. As for
610 version 1.6 and higher it is in the distribution.
611
612 Added a function that prints the pbs python interface version. So we
613 can check if we have the right interface version:
614 import pbs
615 print pbs.version()
616 
617
618=========== Version 2.2
619
620In this release we can query the pbs_mom daemon with the aid of
621resource management functions:
622   pbs.openrm()
623   pbs.closerm()
624   pbs.downrm()
625   pbs.configrm()
626   pbs.addreq()
627   pbs.allreq()
628   pbs.flushreq()
629   pbs.activereq()
630   pbs.fullresp()
631   pbs.getreq()
632
633There is also a python function:
634   pbs.get_mom_values(id, [list]):
635     id   - connection id of pbs.openrm(node-name, port-number)
636     list - Is a optional paramter. A list of resource keywords
637
638   If 'list' is not supplied then this function will get the values
639   for the standard resource keywords, eg uname, loadave, .. + 'arch'
640   depended keywords. The arch feature is only implemented for linux,
641   but it can easily be ported to other oses.
642
643   if 'list' is supplied then we only fetch values for the keywords
644   that are in the list.
645
646   The function returns a dictonary. The keys are the resource keywords.
647
648   See 'resmom_info.py' for a example of this new feature.
649
650=========== Version 2.0
651
652The previous version was a simple interface above the PBS C API LIB.  This
653interface has gone a major changes. The most noticeable change is  that
654the functions accept/return Python lists instead of C-lists.  So you can now
655use standard Python syntax for manipulating Python lists. If you are familar
656with the Python syntax then you will appreciate this new interface.
657
658NOTE:
659  This interface is NOT compatible with the old one
660
661Here are some guidelines to convert your code to the new one:
662
663The constructors for the struct has gone a major change:
664  old code:
665    temp = pbs.new_attrl();
666    attrl_p = pbs.attrlPtr(temp)
667    attrl_p.name = 'state'
668
669  new_code:
670     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
671     attr_l[0].name = 'state'
672
673
674The pbs_stat functions returns Python lists instead of C-lists. There is
675NO next field anymore:
676  old_code:
677    temp = pbs.new_attrl();
678    attrl_p = pbs.attrlPtr(temp)
679    attrl_p.this = 'NULL'
680
681    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
682
683    while nodes.this != 'NULL':
684      print nodes.name
685
686      node_attrl = nodes.attribs
687      while node_attrl.this != 'NULL':
688        print '\t', node_attrl.name, '=', node_attrl.value
689        node_attrl = node_attrl.next
690
691    nodes = nodes.next
692
693
694  new_code:
695    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
696    for node in nodes:
697      print node.name
698      for attrib in node.attribs:
699        print '\t', attrib.name, '=', attrib.value
700
701Another advantage is you can use the print statement to show the
702connect of attrl and attropl classes:
703   attr_l = pbs.new_attrl(2)
704   attr_l[0].name = 'bas'
705   attr_l[0].value = 'van der Vlies'
706
707   print  attr_l[0]
708   >> (bas,,van der Vlies)
709   
710I hope these examples illustrate the changes. If you specify a wrong type
711for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.