source: tags/4.3.0/CHANGES @ 324

Last change on this file since 324 was 266, checked in by bas, 13 years ago

some reformatting
  • Property svn:keywords set to Id
File size: 20.3 KB
Line 
1=========== 4.3.0
2 * new_rack_pbsmon.py: For backward compatibility added -a/--all option.
3   Has the same functionality as -s/--summary
4   Author: Bas van der Vlies
5
6 * Added some license info to debian package and include GPL and LGPL 3.0
7   files.
8   Suggested by: W. Trevor King <wking at drexel dot edu>
9
10 * PBSQuery.py: Added support for new feature in node status field:
11    * message=EVENT:<key=value:>+
12    * message=ERROR <text>
13
14  Added the following syntax for node:
15        * node['error']
16        * node['event']
17
18  eg: node: n1
19    state = free
20    np = 8
21    properties = ib,switch1,highmem
22    ntype = cluster
23    jobs = 0/567403.sara.nl, 1/567403.sara.nl
24    status = ..,loadave=0.00,message=EVENT:sample.time=1288864220.003:cputotals.user=0:iconnect.pktout=0,netload=3487600394,state=free,...
25
26  example code:
27        p = PBSQuery()
28        node = p.getnode(n1)
29        print node.event['sample.time']
30        print node['event']['sample.time']
31
32   Requested by: Roy Dragseth <roy dot dragseth add uit dotno>
33   Author: Bas van der Vlies
34
35=========== 4.2.0
36 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
37   (jsbronder add gentoo dot org) for the information and
38   explanation.
39
40 * examples/pbsmon.py. Patched so it also works for torque 2.5
41   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
42   Applied by: Bas van der Vlies
43
44 * setup.py.in changed determine which torque version is installed
45   so we also support snapshots.
46   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
47   Applied by: Bas van der Vlies
48
49 * PBSQuery:
50   - get_nodes function for jobs now works for old and new datastructure and
51     gives correct results if unique=True
52     Reported by + idea for patch: Shunjie Lau (eijnuhs add gmail dot com)
53     Author: Bas van der Vlies
54
55 * Added AdvanceParser module. new_rack_pbsmon.py and sara_nodes.py make use of
56   this module
57   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
58
59 * examples/new_rack_pbsmon.py:
60     - Better specification for node expression
61     - added a new option -l/--list to list the state of a nodes and which job
62       is running on it, eg:
63             new_rack_pbsmon.py - l gb-r[1-3]n[3-6] gb-r4[n1-20]
64
65   Authors: Dennis Stam & Bas van der Vlies
66   Applied by: Bas van der Vlies
67
68 * examples/sara_nodes.py:
69   An utility to set/clear notes for multiple nodes, see ..
70   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
71
72 * setup.py.in:
73   Added support for runtime-libraries when torque is installed in a non-default
74   location, closes #22
75   Author: Mattias (slabanja at chalmers dot se)
76   Applied by: Bas van der Vlies
77
78 * Changed build system for debian
79   Authors: Dennis Stam & Bas van der Vlies
80
81=========== 4.1.2
82 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
83   structure.
84   Author: Bas van der Vlies
85
86 * PBSQuery:
87     - fixed and error in get_jobs function for nodes. Did not
88       support new data structure and uniq did not work.
89       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
90
91=========== 4.1.0
92 * Delete function pbs.version() replaced by
93    * pbs.version
94        This is a string variable
95    * pbs.version_info
96        Is a tuple variable. So we can use it in scripts
97
98   This analogue with sys module, closes #20
99
100   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
101   Author: Bas van der Vlies
102
103 * More robust make/configure/setup.py
104   Reported by:
105        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
106        - Shunjie Lau <eijnuhs add gmail dot com>
107   Author: Bas van der Vlies
108
109
110 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
111   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
112   Author: Bas van der Vlies
113   
114=========== 4.0.2
115  * Quick fix so we do not get any undefined symbols. Forgot to
116    add the torque library when we compile the package.
117    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
118    Author: Bas van der Vlies
119
120=========== 4.0.1
121  * Fixed a bug in setting the right library path  for compiling.
122    Reported by: Chad Vizino ( vizino add psc dot edu )
123    Author: Bas van der Vlies
124   
125=========== 4.0.0
126  * pbs.py added some new functions for 2.4 version:
127    - pbs_fbserver(void);
128    - pbs_get_server_list(void);
129    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
130    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
131    - pbs_checkpointjob(int connect, char *job_id, char *extend);
132    - log_ext(int,char *,char *,int);
133    - log_init(char *, char *);
134    - log_remove_old(char *,unsigned long);
135
136    Note: version 2.1.X is still supported
137
138  * autotools environment
139    - Added support for finding pbs-config (borrowed for mpiexec source)
140    - Detecting which version of torque is installed
141    - only support installations that have pbs-config installed.
142    - Removed old code for installations that do not a have pbs-config
143      installed.
144
145  * PBSQuery.py
146    - There was a bug in the new data interface (Resource_List for jobs)
147      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
148      Applied by with some minor changes: Bas van der Vlies
149
150    - The new data structure is the default. You can switch back to the old
151      one with:
152            p = PBSQuery()
153            p.old_data_structure()
154
155      Author: Bas van der Vlies
156
157   * examples/ha_server.py
158     An example how to use the High Availabilty torque functions
159     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
160
161=========== 3.6.0
162  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
163    Reported by: Dennis Stam
164    Fixed by: Bas van der Vlies
165
166=========== 3.5.0
167  * PBSQuery
168        The class functions of node, job and queue support old and new data
169        structure.
170
171        Changed the behaviour of the new data stucture, We can use it as
172        dictionary and as class attribute, this is equivalent, eg:
173          - print node['np'] and print node.np
174
175        for a node we parse the 'status' line and split on '=' char, You now can
176        use these statements, eg
177          - print node.status.arch     (node['status'].arch or node['status']['arch'])
178          - print node.status.nsession
179
180        for a job we parse the 'Variable_List' line and split on '=' char, You now can
181        use the statements, eg:
182          - print job.Variable_List.PBS_O_WORKDIR
183          - print job.Variable_List.PBS_O_HOME
184
185        For more info see examples/new_interface.py
186
187    Author: Bas van der Vlies
188
189 * new_rack_pbsmon.py
190        Rewrite to new data structure and automatically determine how many nodes
191        and racks cluster has and skip printing of empty racks (default), use -w/--wide
192        for old behaviour.
193
194   Author: Bas van der Vlies
195
196
197=========== 3.2.0
198        - PBSQuery:
199          New data structure. In the old structure it is a dictionary
200          with a value and the value is a string. This is changed
201          that dictionary values are now of type list or dictionary depends
202          on the value of the keyword, eg for a node:
203            - np = 2:
204              * node['np'] = [ '2' ] 
205
206            - properties = cores2, mem4gb, parallel
207              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
208
209            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
210              * node['status']['arch'] = [ 'x86_64' ]
211              * node['status']['sessions'] = [ '222599' ]
212              * ...
213
214         The data structure is activated by the function:
215          * new_data_structure()
216
217        In a future release it will be come the default.
218        example:
219                p = PBSQuery()
220                p.new_data_structure()
221
222                nodes = p.getnodes()
223                print nodes.np, nodes['np']
224
225        Author: Bas van der Vlies
226
227        - PBSQuery
228          For old and new data structure we now can use another syntax:
229           * node['np'] and node.np are equivalent
230
231          This new syntax works for all keywords.
232
233        Author: Bas van der Vlies
234
235        - Added iter object for job, node, queue and server objects, eg:
236          node = p.getnode('gb-r1n1')
237          print node.name
238          for attrib in node:
239                print '%\t%s = %s' %(attrib, node[attrib])
240        Author: Bas van der Vlies
241
242        - fixed an error in getnode, getqueue and getjob, return empty
243          dictionary if not found
244        Author: Bas van der Vlies
245       
246        - New build system for rpm packages, make -f Makefile.rpm
247          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
248
249        Applied: Bas van der Vlies
250
251=========== 3.0.1
252        - PBSQuery
253                * Removed a testing code line
254                Reported By: sam @ vpac dot org
255                Fixed by: Bas van der Vlies
256
257=========== 3.0.0
258        - PBSQuery
259                * Cleaned up some old lines of code which didn't do anything.
260                * Changed the returnvalue of getqueue, getnode and getjob. Now
261                  they return an instance of the object, instead of a dict.
262                  This is API change so we bumped the version to 3.0.0
263                * Now the getnode-function accepts the short name as well as
264                  the long name as an argument.
265
266                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
267
268        - pbs.py, _pbs.so, PBSQuery.py
269                * Fixed a memory leak when used in daemon mode
270                Author: Bas van der Vlies <basv@sara.nl>
271
272=========== 2.9.8
273        - pbs_python.spec file patch added libdir and python defines so it
274          will build for RHEL5.1/CentOS 5.1
275          by        : Michael Sternberg <sternberg add anl dot gov>
276          applied by: Bas van der Vlies
277
278        - Updated the header files for pbs_python. So it can use some new
279          defines variables/defines, eg:
280                * ATTR_NODE_status
281                * ATTR_NODE_note
282                * ...
283          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
284
285        - Two new functions to log.h:
286                log_roll(int size);
287                log_size(void);
288          Added by: Bas van der Vlies
289
290        - We now have proper destructors (free memory) in SWIG for attrl and
291          attropl attributes.
292          Added by: Bas van der Vlies
293
294        - PBSQuery
295                * Added getnodes_with_property() function. This will get all nodes with
296                  a certain property, eg:
297                   - get all nodes with property 'cores8' and return all attributes:
298                      getnodes_with_property('cores8')
299               
300                   - get all nodes with property 'cores2' and only return the
301                     attributes 'state' and 'properties':
302                      attrl = [ 'state', 'properties' ]
303                      getnodes_with_property('cores2', attrl)
304               
305                     Only for torque version > 2.1.0
306             
307                Added by: Bas van der Vlies
308
309                * Added to class node the function
310                 - get_jobs(self, unique=None)
311                   Returns a list of the currently running job-id('s) on the node.
312                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
313
314                * Added get_server_name() function.
315                  This will return the PBS-server's name.
316                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
317
318                * Added to class _PBSobject the function
319                  - uniq(self, list)
320                    Filters out unique items of a list.
321                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
322
323                * Added to class job the function
324                  - get_nodes(self, unique=None)
325                    Returns a list of the nodes which run the job.
326                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
327
328=========== Version 2.9.4
329        -  Function pbs.pbs_geterrmsg does not exits any more is
330           replaced by pbs.error():
331                errno, text = pbs.error()
332                print errno, text
333           Reported by: pk at q-leap dot com
334           Fixed by   : Bas van der Vlies
335
336        - PBSQuery.py
337           * Added get_version() for server class, returns the version of
338             the batch server
339             Added by    : Bas van der Vlies
340
341           * Added get_nodes() for job class, returns a list of nodes on
342             which the job is run
343             Added by    : Bas van der Vlies
344
345           * PBSQuery.py rewrite make use of UserDict module.
346             Suggested by: Ramon Bastiaans
347             Fixed by    : Bas van der Vlies
348
349=========== Version 2.9.2
350        - The function pbs_rescquery() did not work. Fixed it
351        - The function avail(), did not work fixed it
352        - The functions pbs_rescreserve and pbs_rescrelease did not work
353          fixed it
354
355        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
356          nodes was wrong and state down has topmost priority
357
358=========== Version 2.9.0
359        - Added support for new torque build system 2.1.0
360        - Added new_rack_pbsmon.py
361          Authors: Walter de Jong & Bas van der Vlies
362
363=========== Version 2.8.2
364
365Configure has a new option:
366        --with-pbsdir=PATH
367
368If set it will use PATH specified as argument to
369find the pbs/torque libraries. If unset it will use
370some useful defaults to find the libaries.
371Suggested by: Troy Baer
372
373PBSQuery.py:
374        - If we can not make a connection with the server
375          then raise an exception and let the programmer
376          decides what to do.
377
378pbs_python.spec:
379        - It can now handle mode bits
380          Author: Martin Pels from SARA
381       
382=========== Version 2.8.0
383
384setup.py:
385        - Added /usr/lib to PBS_LIB_DIR
386
387pbs.py:
388        - Added new function "pbs.error()". This function checks if
389          an error has occured with a pbs function, eg:
390            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
391            error_number, err_txt = pbs.error()
392            if error_number:
393                print error_number, err_txt
394
395          prints the following message if script A.tsk does not exists:
396            15042 (qsub) cannot access script file
397
398pbs_python.spec:
399        - to make rpm packages from the source package, initial version
400        Author: Martin Pels from SARA
401
402README:
403        - Now requires python 2.1 and higher
404        - Explain how to build DEBIAN package
405
406=========== Version 2.7.10
407
408PBSQuery.py:
409        Fixed an error. You always got all resources from eg nodes
410        even when you supplied an attribute list where you only
411        requested the 'state' of the node.
412
413        Updated the inline python documentation
414
415=========== Version 2.7.9
416
417Fixed an error the previous version was in DEBUG mode. So
418you get an lot undesired output on the screen.
419
420debian package now also generated a package with the write
421version info.
422
423=========== Version 2.7.8
424
425Setup.py:
426        Can also handle ROCKS installations + the version
427        info is now the same as pbs.py
428        Thanks to: Roy Dragseth
429
430=========== Version 2.7.7
431
432PBSQuery module:
433  It can now be used in daemon programs. The pbs_server closes
434  the connection after an certain amount of time. This is fixed,
435  before every query there is an new connection and if the query
436  is finished the connection is closed.
437 
438=========== Version 2.7.6
439
440Now setup.py can also handle old Oscar Installations.
441Thanks To: Robin Humble
442
443Fixed an print statement fix in new_interface.py:
444Thanks To: Robin Humble
445
446Made some improvements for PBSQuery module:
447        - Updated the documentation
448        - Added the has_key() function to all
449          PBSObjects. So that the behaviour is similar to
450          an dictionary
451          Suggested By: Ramon Bastiaans
452       
453=========== Version 2.7.5
454 Added PBSQuery module.  This module requires pbs.py and it a simple module
455 for querying the pbs server. The documentation is in the module. Use for
456 example ipython to read it.
457
458 Written By: Roy Dragseth
459             Bas van der Vlies
460
461 eg: ( see also examples/new_interface.py)
462
463 from PBSQuery import PBSQuery
464 p = PBSQuery()
465
466 jobs = p.getjobs()
467 for name, job in jobs.items()
468        print job
469
470=========== Version 2.7.4
471 New versions fixes an bug in function pbs.pbs_statnode().
472 Reported by: Keith Poirier
473
474 Configure support:
475 Contributed By: Yaroslav Halchenko
476
477 Debian package support:
478 Contributed By: Yaroslav Halchenko
479
480 Added Support for Debuging the interface, You must edit pbs_wrap.c
481 and search for SARA_DEBUG:
482        #define SARA_DEBUG 1
483 Implemented by: Ramon Bastiaans
484
485 examples/pbsmon.py:
486        - Fixed an error when regex fails to determine node number,
487          no status was displayed
488        - It will now display and 'j' if the node is free for the batch
489          system and a job runs on the node (SMP-systems).
490 Changed by: Bas van der Vlies
491
492 examples/rack_pbsmon.py:
493        - An pbsmon that display node info per rack
494        - Edit the rack_pbsmon.py to adjust the values. It requires
495          that hostname contains rack and node id's,
496          eg: gb-r<number>n<number>
497 Contributed by: Walter de Jong
498
499=========== Version 2.7.3
500 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
501 Resource and QUEing manager).
502
503 The interface support OPENPBS and TORQUE.
504
505=========== Version 2.7.2
506 Some minor changes to include files. Now the Scalable PBS keywords
507 are also supported ( server attribute names: node_ping_rate and
508 node_check_rate). This does not interfere with the openpbs software.
509
510 pbsmon.py could not handle an one node cluster. Fixed it.
511
512=========== Version 2.7
513 Forgot to wrap the pbs_statfree() function. So we could not free
514 allocated memory from functions that return 'struct batch_status *'
515 like pbs_statjob().
516
517 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
518 Daniel Olson
519
520=========== Version 2.6.1
521 Setup.py now checks if all openpbs libraries are installed to compile the
522 package.
523
524=========== Version 2.6
525
526 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
527 function. This bug prevented to pass 'struct attrl' variables to this
528 function. Thanks to Evelyn Shiu for reporting this bug.
529
530
531=========== Version 2.5
532 Added the OpenPbs logging functions (log.h and liblog.a). There
533 is an example in the examples directory: logpbs.py
534
535 Fixed a bug in examples/resmom_info.py used the default port for
536 pbs_resmon if getservbyname fails.
537
538 Fixed a bug in resmom code. If the user has no permissions to query
539 the pbs_mom daemon a empty string is returned. Now we check for
540 the empty string. So we do not get a python exception.
541
542 Fixed a bug in setup.py forgot the exit statement when we did not
543 find the pbs libraries.
544
545=========== Version 2.3
546 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
547
548 Removed some obsolete code.
549
550 As for now we make use of the distutils module. So the package
551 is automatically compiled and installed in proper directories, usage:
552    python setup.py install
553 This requires that the package 'distutils' must be installed. As for
554 version 1.6 and higher it is in the distribution.
555
556 Added a function that prints the pbs python interface version. So we
557 can check if we have the right interface version:
558 import pbs
559 print pbs.version()
560 
561
562=========== Version 2.2
563
564In this release we can query the pbs_mom daemon with the aid of
565resource management functions:
566   pbs.openrm()
567   pbs.closerm()
568   pbs.downrm()
569   pbs.configrm()
570   pbs.addreq()
571   pbs.allreq()
572   pbs.flushreq()
573   pbs.activereq()
574   pbs.fullresp()
575   pbs.getreq()
576
577There is also a python function:
578   pbs.get_mom_values(id, [list]):
579     id   - connection id of pbs.openrm(node-name, port-number)
580     list - Is a optional paramter. A list of resource keywords
581
582   If 'list' is not supplied then this function will get the values
583   for the standard resource keywords, eg uname, loadave, .. + 'arch'
584   depended keywords. The arch feature is only implemented for linux,
585   but it can easily be ported to other oses.
586
587   if 'list' is supplied then we only fetch values for the keywords
588   that are in the list.
589
590   The function returns a dictonary. The keys are the resource keywords.
591
592   See 'resmom_info.py' for a example of this new feature.
593
594=========== Version 2.0
595
596The previous version was a simple interface above the PBS C API LIB.  This
597interface has gone a major changes. The most noticeable change is  that
598the functions accept/return Python lists instead of C-lists.  So you can now
599use standard Python syntax for manipulating Python lists. If you are familar
600with the Python syntax then you will appreciate this new interface.
601
602NOTE:
603  This interface is NOT compatible with the old one
604
605Here are some guidelines to convert your code to the new one:
606
607The constructors for the struct has gone a major change:
608  old code:
609    temp = pbs.new_attrl();
610    attrl_p = pbs.attrlPtr(temp)
611    attrl_p.name = 'state'
612
613  new_code:
614     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
615     attr_l[0].name = 'state'
616
617
618The pbs_stat functions returns Python lists instead of C-lists. There is
619NO next field anymore:
620  old_code:
621    temp = pbs.new_attrl();
622    attrl_p = pbs.attrlPtr(temp)
623    attrl_p.this = 'NULL'
624
625    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
626
627    while nodes.this != 'NULL':
628      print nodes.name
629
630      node_attrl = nodes.attribs
631      while node_attrl.this != 'NULL':
632        print '\t', node_attrl.name, '=', node_attrl.value
633        node_attrl = node_attrl.next
634
635    nodes = nodes.next
636
637
638  new_code:
639    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
640    for node in nodes:
641      print node.name
642      for attrib in node.attribs:
643        print '\t', attrib.name, '=', attrib.value
644
645Another advantage is you can use the print statement to show the
646connect of attrl and attropl classes:
647   attr_l = pbs.new_attrl(2)
648   attr_l[0].name = 'bas'
649   attr_l[0].value = 'van der Vlies'
650
651   print  attr_l[0]
652   >> (bas,,van der Vlies)
653   
654I hope these examples illustrate the changes. If you specify a wrong type
655for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.