source: trunk/CHANGES @ 262

Last change on this file since 262 was 262, checked in by bas, 14 years ago

added -a/--all option for backwards compatibility
  • Property svn:keywords set to Id
File size: 19.4 KB
Line 
1=========== 4.X.X
2 * new_rack_pbsmon.py: For backward compatibility added -a/--all option.
3   Has the same functionality as -s/--summary
4   Author: Bas van der Vlies
5
6=========== 4.2.0
7 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
8   (jsbronder add gentoo dot org) for the information and
9   explanation.
10
11 * examples/pbsmon.py. Patched so it also works for torque 2.5
12   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
13   Applied by: Bas van der Vlies
14
15 * setup.py.in changed determine which torque version is installed
16   so we also support snapshots.
17   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
18   Applied by: Bas van der Vlies
19
20 * PBSQuery:
21   - get_nodes function for jobs now works for old and new datastructure and
22     gives correct results if unique=True
23     Reported by + idea for patch: Shunjie Lau (eijnuhs add gmail dot com)
24     Author: Bas van der Vlies
25
26 * Added AdvanceParser module. new_rack_pbsmon.py and sara_nodes.py make use of
27   this module
28   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
29
30 * examples/new_rack_pbsmon.py:
31     - Better specification for node expression
32     - added a new option -l/--list to list the state of a nodes and which job
33       is running on it, eg:
34             new_rack_pbsmon.py - l gb-r[1-3]n[3-6] gb-r4[n1-20]
35
36   Authors: Dennis Stam & Bas van der Vlies
37   Applied by: Bas van der Vlies
38
39 * examples/sara_nodes.py:
40   An utility to set/clear notes for multiple nodes, see ..
41   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
42
43 * setup.py.in:
44   Added support for runtime-libraries when torque is installed in a non-default
45   location, closes #22
46   Author: Mattias (slabanja at chalmers dot se)
47   Applied by: Bas van der Vlies
48
49 * Changed build system for debian
50   Authors: Dennis Stam & Bas van der Vlies
51
52=========== 4.1.2
53 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
54   structure.
55   Author: Bas van der Vlies
56
57 * PBSQuery:
58     - fixed and error in get_jobs function for nodes. Did not
59       support new data structure and uniq did not work.
60       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
61
62=========== 4.1.0
63 * Delete function pbs.version() replaced by
64    * pbs.version
65        This is a string variable
66    * pbs.version_info
67        Is a tuple variable. So we can use it in scripts
68
69   This analogue with sys module, closes #20
70
71   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
72   Author: Bas van der Vlies
73
74 * More robust make/configure/setup.py
75   Reported by:
76        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
77        - Shunjie Lau <eijnuhs add gmail dot com>
78   Author: Bas van der Vlies
79
80
81 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
82   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
83   Author: Bas van der Vlies
84   
85=========== 4.0.2
86  * Quick fix so we do not get any undefined symbols. Forgot to
87    add the torque library when we compile the package.
88    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
89    Author: Bas van der Vlies
90
91=========== 4.0.1
92  * Fixed a bug in setting the right library path  for compiling.
93    Reported by: Chad Vizino ( vizino add psc dot edu )
94    Author: Bas van der Vlies
95   
96=========== 4.0.0
97  * pbs.py added some new functions for 2.4 version:
98    - pbs_fbserver(void);
99    - pbs_get_server_list(void);
100    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
101    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
102    - pbs_checkpointjob(int connect, char *job_id, char *extend);
103    - log_ext(int,char *,char *,int);
104    - log_init(char *, char *);
105    - log_remove_old(char *,unsigned long);
106
107    Note: version 2.1.X is still supported
108
109  * autotools environment
110    - Added support for finding pbs-config (borrowed for mpiexec source)
111    - Detecting which version of torque is installed
112    - only support installations that have pbs-config installed.
113    - Removed old code for installations that do not a have pbs-config
114      installed.
115
116  * PBSQuery.py
117    - There was a bug in the new data interface (Resource_List for jobs)
118      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
119      Applied by with some minor changes: Bas van der Vlies
120
121    - The new data structure is the default. You can switch back to the old
122      one with:
123            p = PBSQuery()
124            p.old_data_structure()
125
126      Author: Bas van der Vlies
127
128   * examples/ha_server.py
129     An example how to use the High Availabilty torque functions
130     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
131
132=========== 3.6.0
133  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
134    Reported by: Dennis Stam
135    Fixed by: Bas van der Vlies
136
137=========== 3.5.0
138  * PBSQuery
139        The class functions of node, job and queue support old and new data
140        structure.
141
142        Changed the behaviour of the new data stucture, We can use it as
143        dictionary and as class attribute, this is equivalent, eg:
144          - print node['np'] and print node.np
145
146        for a node we parse the 'status' line and split on '=' char, You now can
147        use these statements, eg
148          - print node.status.arch     (node['status'].arch or node['status']['arch'])
149          - print node.status.nsession
150
151        for a job we parse the 'Variable_List' line and split on '=' char, You now can
152        use the statements, eg:
153          - print job.Variable_List.PBS_O_WORKDIR
154          - print job.Variable_List.PBS_O_HOME
155
156        For more info see examples/new_interface.py
157
158    Author: Bas van der Vlies
159
160 * new_rack_pbsmon.py
161        Rewrite to new data structure and automatically determine how many nodes
162        and racks cluster has and skip printing of empty racks (default), use -w/--wide
163        for old behaviour.
164
165   Author: Bas van der Vlies
166
167
168=========== 3.2.0
169        - PBSQuery:
170          New data structure. In the old structure it is a dictionary
171          with a value and the value is a string. This is changed
172          that dictionary values are now of type list or dictionary depends
173          on the value of the keyword, eg for a node:
174            - np = 2:
175              * node['np'] = [ '2' ] 
176
177            - properties = cores2, mem4gb, parallel
178              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
179
180            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
181              * node['status']['arch'] = [ 'x86_64' ]
182              * node['status']['sessions'] = [ '222599' ]
183              * ...
184
185         The data structure is activated by the function:
186          * new_data_structure()
187
188        In a future release it will be come the default.
189        example:
190                p = PBSQuery()
191                p.new_data_structure()
192
193                nodes = p.getnodes()
194                print nodes.np, nodes['np']
195
196        Author: Bas van der Vlies
197
198        - PBSQuery
199          For old and new data structure we now can use another syntax:
200           * node['np'] and node.np are equivalent
201
202          This new syntax works for all keywords.
203
204        Author: Bas van der Vlies
205
206        - Added iter object for job, node, queue and server objects, eg:
207          node = p.getnode('gb-r1n1')
208          print node.name
209          for attrib in node:
210                print '%\t%s = %s' %(attrib, node[attrib])
211        Author: Bas van der Vlies
212
213        - fixed an error in getnode, getqueue and getjob, return empty
214          dictionary if not found
215        Author: Bas van der Vlies
216       
217        - New build system for rpm packages, make -f Makefile.rpm
218          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
219
220        Applied: Bas van der Vlies
221
222=========== 3.0.1
223        - PBSQuery
224                * Removed a testing code line
225                Reported By: sam @ vpac dot org
226                Fixed by: Bas van der Vlies
227
228=========== 3.0.0
229        - PBSQuery
230                * Cleaned up some old lines of code which didn't do anything.
231                * Changed the returnvalue of getqueue, getnode and getjob. Now
232                  they return an instance of the object, instead of a dict.
233                  This is API change so we bumped the version to 3.0.0
234                * Now the getnode-function accepts the short name as well as
235                  the long name as an argument.
236
237                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
238
239        - pbs.py, _pbs.so, PBSQuery.py
240                * Fixed a memory leak when used in daemon mode
241                Author: Bas van der Vlies <basv@sara.nl>
242
243=========== 2.9.8
244        - pbs_python.spec file patch added libdir and python defines so it
245          will build for RHEL5.1/CentOS 5.1
246          by        : Michael Sternberg <sternberg add anl dot gov>
247          applied by: Bas van der Vlies
248
249        - Updated the header files for pbs_python. So it can use some new
250          defines variables/defines, eg:
251                * ATTR_NODE_status
252                * ATTR_NODE_note
253                * ...
254          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
255
256        - Two new functions to log.h:
257                log_roll(int size);
258                log_size(void);
259          Added by: Bas van der Vlies
260
261        - We now have proper destructors (free memory) in SWIG for attrl and
262          attropl attributes.
263          Added by: Bas van der Vlies
264
265        - PBSQuery
266                * Added getnodes_with_property() function. This will get all nodes with
267                  a certain property, eg:
268                   - get all nodes with property 'cores8' and return all attributes:
269                      getnodes_with_property('cores8')
270               
271                   - get all nodes with property 'cores2' and only return the
272                     attributes 'state' and 'properties':
273                      attrl = [ 'state', 'properties' ]
274                      getnodes_with_property('cores2', attrl)
275               
276                     Only for torque version > 2.1.0
277             
278                Added by: Bas van der Vlies
279
280                * Added to class node the function
281                 - get_jobs(self, unique=None)
282                   Returns a list of the currently running job-id('s) on the node.
283                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
284
285                * Added get_server_name() function.
286                  This will return the PBS-server's name.
287                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
288
289                * Added to class _PBSobject the function
290                  - uniq(self, list)
291                    Filters out unique items of a list.
292                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
293
294                * Added to class job the function
295                  - get_nodes(self, unique=None)
296                    Returns a list of the nodes which run the job.
297                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
298
299=========== Version 2.9.4
300        -  Function pbs.pbs_geterrmsg does not exits any more is
301           replaced by pbs.error():
302                errno, text = pbs.error()
303                print errno, text
304           Reported by: pk at q-leap dot com
305           Fixed by   : Bas van der Vlies
306
307        - PBSQuery.py
308           * Added get_version() for server class, returns the version of
309             the batch server
310             Added by    : Bas van der Vlies
311
312           * Added get_nodes() for job class, returns a list of nodes on
313             which the job is run
314             Added by    : Bas van der Vlies
315
316           * PBSQuery.py rewrite make use of UserDict module.
317             Suggested by: Ramon Bastiaans
318             Fixed by    : Bas van der Vlies
319
320=========== Version 2.9.2
321        - The function pbs_rescquery() did not work. Fixed it
322        - The function avail(), did not work fixed it
323        - The functions pbs_rescreserve and pbs_rescrelease did not work
324          fixed it
325
326        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
327          nodes was wrong and state down has topmost priority
328
329=========== Version 2.9.0
330        - Added support for new torque build system 2.1.0
331        - Added new_rack_pbsmon.py
332          Authors: Walter de Jong & Bas van der Vlies
333
334=========== Version 2.8.2
335
336Configure has a new option:
337        --with-pbsdir=PATH
338
339If set it will use PATH specified as argument to
340find the pbs/torque libraries. If unset it will use
341some useful defaults to find the libaries.
342Suggested by: Troy Baer
343
344PBSQuery.py:
345        - If we can not make a connection with the server
346          then raise an exception and let the programmer
347          decides what to do.
348
349pbs_python.spec:
350        - It can now handle mode bits
351          Author: Martin Pels from SARA
352       
353=========== Version 2.8.0
354
355setup.py:
356        - Added /usr/lib to PBS_LIB_DIR
357
358pbs.py:
359        - Added new function "pbs.error()". This function checks if
360          an error has occured with a pbs function, eg:
361            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
362            error_number, err_txt = pbs.error()
363            if error_number:
364                print error_number, err_txt
365
366          prints the following message if script A.tsk does not exists:
367            15042 (qsub) cannot access script file
368
369pbs_python.spec:
370        - to make rpm packages from the source package, initial version
371        Author: Martin Pels from SARA
372
373README:
374        - Now requires python 2.1 and higher
375        - Explain how to build DEBIAN package
376
377=========== Version 2.7.10
378
379PBSQuery.py:
380        Fixed an error. You always got all resources from eg nodes
381        even when you supplied an attribute list where you only
382        requested the 'state' of the node.
383
384        Updated the inline python documentation
385
386=========== Version 2.7.9
387
388Fixed an error the previous version was in DEBUG mode. So
389you get an lot undesired output on the screen.
390
391debian package now also generated a package with the write
392version info.
393
394=========== Version 2.7.8
395
396Setup.py:
397        Can also handle ROCKS installations + the version
398        info is now the same as pbs.py
399        Thanks to: Roy Dragseth
400
401=========== Version 2.7.7
402
403PBSQuery module:
404  It can now be used in daemon programs. The pbs_server closes
405  the connection after an certain amount of time. This is fixed,
406  before every query there is an new connection and if the query
407  is finished the connection is closed.
408 
409=========== Version 2.7.6
410
411Now setup.py can also handle old Oscar Installations.
412Thanks To: Robin Humble
413
414Fixed an print statement fix in new_interface.py:
415Thanks To: Robin Humble
416
417Made some improvements for PBSQuery module:
418        - Updated the documentation
419        - Added the has_key() function to all
420          PBSObjects. So that the behaviour is similar to
421          an dictionary
422          Suggested By: Ramon Bastiaans
423       
424=========== Version 2.7.5
425 Added PBSQuery module.  This module requires pbs.py and it a simple module
426 for querying the pbs server. The documentation is in the module. Use for
427 example ipython to read it.
428
429 Written By: Roy Dragseth
430             Bas van der Vlies
431
432 eg: ( see also examples/new_interface.py)
433
434 from PBSQuery import PBSQuery
435 p = PBSQuery()
436
437 jobs = p.getjobs()
438 for name, job in jobs.items()
439        print job
440
441=========== Version 2.7.4
442 New versions fixes an bug in function pbs.pbs_statnode().
443 Reported by: Keith Poirier
444
445 Configure support:
446 Contributed By: Yaroslav Halchenko
447
448 Debian package support:
449 Contributed By: Yaroslav Halchenko
450
451 Added Support for Debuging the interface, You must edit pbs_wrap.c
452 and search for SARA_DEBUG:
453        #define SARA_DEBUG 1
454 Implemented by: Ramon Bastiaans
455
456 examples/pbsmon.py:
457        - Fixed an error when regex fails to determine node number,
458          no status was displayed
459        - It will now display and 'j' if the node is free for the batch
460          system and a job runs on the node (SMP-systems).
461 Changed by: Bas van der Vlies
462
463 examples/rack_pbsmon.py:
464        - An pbsmon that display node info per rack
465        - Edit the rack_pbsmon.py to adjust the values. It requires
466          that hostname contains rack and node id's,
467          eg: gb-r<number>n<number>
468 Contributed by: Walter de Jong
469
470=========== Version 2.7.3
471 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
472 Resource and QUEing manager).
473
474 The interface support OPENPBS and TORQUE.
475
476=========== Version 2.7.2
477 Some minor changes to include files. Now the Scalable PBS keywords
478 are also supported ( server attribute names: node_ping_rate and
479 node_check_rate). This does not interfere with the openpbs software.
480
481 pbsmon.py could not handle an one node cluster. Fixed it.
482
483=========== Version 2.7
484 Forgot to wrap the pbs_statfree() function. So we could not free
485 allocated memory from functions that return 'struct batch_status *'
486 like pbs_statjob().
487
488 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
489 Daniel Olson
490
491=========== Version 2.6.1
492 Setup.py now checks if all openpbs libraries are installed to compile the
493 package.
494
495=========== Version 2.6
496
497 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
498 function. This bug prevented to pass 'struct attrl' variables to this
499 function. Thanks to Evelyn Shiu for reporting this bug.
500
501
502=========== Version 2.5
503 Added the OpenPbs logging functions (log.h and liblog.a). There
504 is an example in the examples directory: logpbs.py
505
506 Fixed a bug in examples/resmom_info.py used the default port for
507 pbs_resmon if getservbyname fails.
508
509 Fixed a bug in resmom code. If the user has no permissions to query
510 the pbs_mom daemon a empty string is returned. Now we check for
511 the empty string. So we do not get a python exception.
512
513 Fixed a bug in setup.py forgot the exit statement when we did not
514 find the pbs libraries.
515
516=========== Version 2.3
517 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
518
519 Removed some obsolete code.
520
521 As for now we make use of the distutils module. So the package
522 is automatically compiled and installed in proper directories, usage:
523    python setup.py install
524 This requires that the package 'distutils' must be installed. As for
525 version 1.6 and higher it is in the distribution.
526
527 Added a function that prints the pbs python interface version. So we
528 can check if we have the right interface version:
529 import pbs
530 print pbs.version()
531 
532
533=========== Version 2.2
534
535In this release we can query the pbs_mom daemon with the aid of
536resource management functions:
537   pbs.openrm()
538   pbs.closerm()
539   pbs.downrm()
540   pbs.configrm()
541   pbs.addreq()
542   pbs.allreq()
543   pbs.flushreq()
544   pbs.activereq()
545   pbs.fullresp()
546   pbs.getreq()
547
548There is also a python function:
549   pbs.get_mom_values(id, [list]):
550     id   - connection id of pbs.openrm(node-name, port-number)
551     list - Is a optional paramter. A list of resource keywords
552
553   If 'list' is not supplied then this function will get the values
554   for the standard resource keywords, eg uname, loadave, .. + 'arch'
555   depended keywords. The arch feature is only implemented for linux,
556   but it can easily be ported to other oses.
557
558   if 'list' is supplied then we only fetch values for the keywords
559   that are in the list.
560
561   The function returns a dictonary. The keys are the resource keywords.
562
563   See 'resmom_info.py' for a example of this new feature.
564
565=========== Version 2.0
566
567The previous version was a simple interface above the PBS C API LIB.  This
568interface has gone a major changes. The most noticeable change is  that
569the functions accept/return Python lists instead of C-lists.  So you can now
570use standard Python syntax for manipulating Python lists. If you are familar
571with the Python syntax then you will appreciate this new interface.
572
573NOTE:
574  This interface is NOT compatible with the old one
575
576Here are some guidelines to convert your code to the new one:
577
578The constructors for the struct has gone a major change:
579  old code:
580    temp = pbs.new_attrl();
581    attrl_p = pbs.attrlPtr(temp)
582    attrl_p.name = 'state'
583
584  new_code:
585     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
586     attr_l[0].name = 'state'
587
588
589The pbs_stat functions returns Python lists instead of C-lists. There is
590NO next field anymore:
591  old_code:
592    temp = pbs.new_attrl();
593    attrl_p = pbs.attrlPtr(temp)
594    attrl_p.this = 'NULL'
595
596    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
597
598    while nodes.this != 'NULL':
599      print nodes.name
600
601      node_attrl = nodes.attribs
602      while node_attrl.this != 'NULL':
603        print '\t', node_attrl.name, '=', node_attrl.value
604        node_attrl = node_attrl.next
605
606    nodes = nodes.next
607
608
609  new_code:
610    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
611    for node in nodes:
612      print node.name
613      for attrib in node.attribs:
614        print '\t', attrib.name, '=', attrib.value
615
616Another advantage is you can use the print statement to show the
617connect of attrl and attropl classes:
618   attr_l = pbs.new_attrl(2)
619   attr_l[0].name = 'bas'
620   attr_l[0].value = 'van der Vlies'
621
622   print  attr_l[0]
623   >> (bas,,van der Vlies)
624   
625I hope these examples illustrate the changes. If you specify a wrong type
626for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.