source: trunk/CHANGES @ 253

Last change on this file since 253 was 253, checked in by bas, 14 years ago

Updated version to 4.2.0 and update CHANGES file
  • Property svn:keywords set to Id
File size: 18.9 KB
Line 
1=========== 4.2.0
2 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
3  (jsbronder add gentoo dot org) for the information and
4  explanation.
5
6 * examples/pbsmon.py. Patched so it also works for torque 2.5
7   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
8   Applied by: Bas van der Vlies
9
10 * setup.py.in changed determine which torque version is installed
11   so we also support snapshots.
12   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
13   Applied by: Bas van der Vlies
14
15 * PBSQuery:
16   - get_nodes function for jobs now works for old and new datastructure and
17     gives correct results if unique=True
18     Author: Bas van der Vlies
19
20 * Added AdvanceParser module. new_rack_pbsmon.py and sara_nodes.py make use of
21   this module
22   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
23
24 * examples/new_rack_pbsmon.py:
25     - Better specification for node expression
26     - added a new option -l/--list to list the state of a nodes and which job
27       is running on it, eg:
28             new_rack_pbsmon.py - l gb-r[1-3]n[3-6] gb-r4[n1-20]
29
30   Author: Dennis Stam & Bas van der Vlies
31   Applied by: Bas van der Vlies
32
33 * examples/sara_nodes.py:
34   An utility to set/clear notes for multiple nodes, see ..
35   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
36
37=========== 4.1.2
38 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
39   structure.
40   Author: Bas van der Vlies
41
42 * PBSQuery:
43     - fixed and error in get_jobs function for nodes. Did not
44       support new data structure and uniq did not work.
45       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
46
47=========== 4.1.0
48 * Delete function pbs.version() replaced by
49    * pbs.version
50        This is a string variable
51    * pbs.version_info
52        Is a tuple variable. So we can use it in scripts
53
54   This analogue with sys module, closes #20
55
56   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
57   Author: Bas van der Vlies
58
59 * More robust make/configure/setup.py
60   Reported by:
61        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
62        - Shunjie Lau <eijnuhs add gmail dot com>
63   Author: Bas van der Vlies
64
65
66 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
67   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
68   Author: Bas van der Vlies
69   
70=========== 4.0.2
71  * Quick fix so we do not get any undefined symbols. Forgot to
72    add the torque library when we compile the package.
73    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
74    Author: Bas van der Vlies
75
76=========== 4.0.1
77  * Fixed a bug in setting the right library path  for compiling.
78    Reported by: Chad Vizino ( vizino add psc dot edu )
79    Author: Bas van der Vlies
80   
81=========== 4.0.0
82  * pbs.py added some new functions for 2.4 version:
83    - pbs_fbserver(void);
84    - pbs_get_server_list(void);
85    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
86    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
87    - pbs_checkpointjob(int connect, char *job_id, char *extend);
88    - log_ext(int,char *,char *,int);
89    - log_init(char *, char *);
90    - log_remove_old(char *,unsigned long);
91
92    Note: version 2.1.X is still supported
93
94  * autotools environment
95    - Added support for finding pbs-config (borrowed for mpiexec source)
96    - Detecting which version of torque is installed
97    - only support installations that have pbs-config installed.
98    - Removed old code for installations that do not a have pbs-config
99      installed.
100
101  * PBSQuery.py
102    - There was a bug in the new data interface (Resource_List for jobs)
103      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
104      Applied by with some minor changes: Bas van der Vlies
105
106    - The new data structure is the default. You can switch back to the old
107      one with:
108            p = PBSQuery()
109            p.old_data_structure()
110
111      Author: Bas van der Vlies
112
113   * examples/ha_server.py
114     An example how to use the High Availabilty torque functions
115     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
116
117=========== 3.6.0
118  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
119    Reported by: Dennis Stam
120    Fixed by: Bas van der Vlies
121
122=========== 3.5.0
123  * PBSQuery
124        The class functions of node, job and queue support old and new data
125        structure.
126
127        Changed the behaviour of the new data stucture, We can use it as
128        dictionary and as class attribute, this is equivalent, eg:
129          - print node['np'] and print node.np
130
131        for a node we parse the 'status' line and split on '=' char, You now can
132        use these statements, eg
133          - print node.status.arch     (node['status'].arch or node['status']['arch'])
134          - print node.status.nsession
135
136        for a job we parse the 'Variable_List' line and split on '=' char, You now can
137        use the statements, eg:
138          - print job.Variable_List.PBS_O_WORKDIR
139          - print job.Variable_List.PBS_O_HOME
140
141        For more info see examples/new_interface.py
142
143    Author: Bas van der Vlies
144
145 * new_rack_pbsmon.py
146        Rewrite to new data structure and automatically determine how many nodes
147        and racks cluster has and skip printing of empty racks (default), use -w/--wide
148        for old behaviour.
149
150   Author: Bas van der Vlies
151
152
153=========== 3.2.0
154        - PBSQuery:
155          New data structure. In the old structure it is a dictionary
156          with a value and the value is a string. This is changed
157          that dictionary values are now of type list or dictionary depends
158          on the value of the keyword, eg for a node:
159            - np = 2:
160              * node['np'] = [ '2' ] 
161
162            - properties = cores2, mem4gb, parallel
163              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
164
165            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
166              * node['status']['arch'] = [ 'x86_64' ]
167              * node['status']['sessions'] = [ '222599' ]
168              * ...
169
170         The data structure is activated by the function:
171          * new_data_structure()
172
173        In a future release it will be come the default.
174        example:
175                p = PBSQuery()
176                p.new_data_structure()
177
178                nodes = p.getnodes()
179                print nodes.np, nodes['np']
180
181        Author: Bas van der Vlies
182
183        - PBSQuery
184          For old and new data structure we now can use another syntax:
185           * node['np'] and node.np are equivalent
186
187          This new syntax works for all keywords.
188
189        Author: Bas van der Vlies
190
191        - Added iter object for job, node, queue and server objects, eg:
192          node = p.getnode('gb-r1n1')
193          print node.name
194          for attrib in node:
195                print '%\t%s = %s' %(attrib, node[attrib])
196        Author: Bas van der Vlies
197
198        - fixed an error in getnode, getqueue and getjob, return empty
199          dictionary if not found
200        Author: Bas van der Vlies
201       
202        - New build system for rpm packages, make -f Makefile.rpm
203          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
204
205        Applied: Bas van der Vlies
206
207=========== 3.0.1
208        - PBSQuery
209                * Removed a testing code line
210                Reported By: sam @ vpac dot org
211                Fixed by: Bas van der Vlies
212
213=========== 3.0.0
214        - PBSQuery
215                * Cleaned up some old lines of code which didn't do anything.
216                * Changed the returnvalue of getqueue, getnode and getjob. Now
217                  they return an instance of the object, instead of a dict.
218                  This is API change so we bumped the version to 3.0.0
219                * Now the getnode-function accepts the short name as well as
220                  the long name as an argument.
221
222                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
223
224        - pbs.py, _pbs.so, PBSQuery.py
225                * Fixed a memory leak when used in daemon mode
226                Author: Bas van der Vlies <basv@sara.nl>
227
228=========== 2.9.8
229        - pbs_python.spec file patch added libdir and python defines so it
230          will build for RHEL5.1/CentOS 5.1
231          by        : Michael Sternberg <sternberg add anl dot gov>
232          applied by: Bas van der Vlies
233
234        - Updated the header files for pbs_python. So it can use some new
235          defines variables/defines, eg:
236                * ATTR_NODE_status
237                * ATTR_NODE_note
238                * ...
239          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
240
241        - Two new functions to log.h:
242                log_roll(int size);
243                log_size(void);
244          Added by: Bas van der Vlies
245
246        - We now have proper destructors (free memory) in SWIG for attrl and
247          attropl attributes.
248          Added by: Bas van der Vlies
249
250        - PBSQuery
251                * Added getnodes_with_property() function. This will get all nodes with
252                  a certain property, eg:
253                   - get all nodes with property 'cores8' and return all attributes:
254                      getnodes_with_property('cores8')
255               
256                   - get all nodes with property 'cores2' and only return the
257                     attributes 'state' and 'properties':
258                      attrl = [ 'state', 'properties' ]
259                      getnodes_with_property('cores2', attrl)
260               
261                     Only for torque version > 2.1.0
262             
263                Added by: Bas van der Vlies
264
265                * Added to class node the function
266                 - get_jobs(self, unique=None)
267                   Returns a list of the currently running job-id('s) on the node.
268                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
269
270                * Added get_server_name() function.
271                  This will return the PBS-server's name.
272                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
273
274                * Added to class _PBSobject the function
275                  - uniq(self, list)
276                    Filters out unique items of a list.
277                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
278
279                * Added to class job the function
280                  - get_nodes(self, unique=None)
281                    Returns a list of the nodes which run the job.
282                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
283
284=========== Version 2.9.4
285        -  Function pbs.pbs_geterrmsg does not exits any more is
286           replaced by pbs.error():
287                errno, text = pbs.error()
288                print errno, text
289           Reported by: pk at q-leap dot com
290           Fixed by   : Bas van der Vlies
291
292        - PBSQuery.py
293           * Added get_version() for server class, returns the version of
294             the batch server
295             Added by    : Bas van der Vlies
296
297           * Added get_nodes() for job class, returns a list of nodes on
298             which the job is run
299             Added by    : Bas van der Vlies
300
301           * PBSQuery.py rewrite make use of UserDict module.
302             Suggested by: Ramon Bastiaans
303             Fixed by    : Bas van der Vlies
304
305=========== Version 2.9.2
306        - The function pbs_rescquery() did not work. Fixed it
307        - The function avail(), did not work fixed it
308        - The functions pbs_rescreserve and pbs_rescrelease did not work
309          fixed it
310
311        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
312          nodes was wrong and state down has topmost priority
313
314=========== Version 2.9.0
315        - Added support for new torque build system 2.1.0
316        - Added new_rack_pbsmon.py
317          Authors: Walter de Jong & Bas van der Vlies
318
319=========== Version 2.8.2
320
321Configure has a new option:
322        --with-pbsdir=PATH
323
324If set it will use PATH specified as argument to
325find the pbs/torque libraries. If unset it will use
326some useful defaults to find the libaries.
327Suggested by: Troy Baer
328
329PBSQuery.py:
330        - If we can not make a connection with the server
331          then raise an exception and let the programmer
332          decides what to do.
333
334pbs_python.spec:
335        - It can now handle mode bits
336          Author: Martin Pels from SARA
337       
338=========== Version 2.8.0
339
340setup.py:
341        - Added /usr/lib to PBS_LIB_DIR
342
343pbs.py:
344        - Added new function "pbs.error()". This function checks if
345          an error has occured with a pbs function, eg:
346            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
347            error_number, err_txt = pbs.error()
348            if error_number:
349                print error_number, err_txt
350
351          prints the following message if script A.tsk does not exists:
352            15042 (qsub) cannot access script file
353
354pbs_python.spec:
355        - to make rpm packages from the source package, initial version
356        Author: Martin Pels from SARA
357
358README:
359        - Now requires python 2.1 and higher
360        - Explain how to build DEBIAN package
361
362=========== Version 2.7.10
363
364PBSQuery.py:
365        Fixed an error. You always got all resources from eg nodes
366        even when you supplied an attribute list where you only
367        requested the 'state' of the node.
368
369        Updated the inline python documentation
370
371=========== Version 2.7.9
372
373Fixed an error the previous version was in DEBUG mode. So
374you get an lot undesired output on the screen.
375
376debian package now also generated a package with the write
377version info.
378
379=========== Version 2.7.8
380
381Setup.py:
382        Can also handle ROCKS installations + the version
383        info is now the same as pbs.py
384        Thanks to: Roy Dragseth
385
386=========== Version 2.7.7
387
388PBSQuery module:
389  It can now be used in daemon programs. The pbs_server closes
390  the connection after an certain amount of time. This is fixed,
391  before every query there is an new connection and if the query
392  is finished the connection is closed.
393 
394=========== Version 2.7.6
395
396Now setup.py can also handle old Oscar Installations.
397Thanks To: Robin Humble
398
399Fixed an print statement fix in new_interface.py:
400Thanks To: Robin Humble
401
402Made some improvements for PBSQuery module:
403        - Updated the documentation
404        - Added the has_key() function to all
405          PBSObjects. So that the behaviour is similar to
406          an dictionary
407          Suggested By: Ramon Bastiaans
408       
409=========== Version 2.7.5
410 Added PBSQuery module.  This module requires pbs.py and it a simple module
411 for querying the pbs server. The documentation is in the module. Use for
412 example ipython to read it.
413
414 Written By: Roy Dragseth
415             Bas van der Vlies
416
417 eg: ( see also examples/new_interface.py)
418
419 from PBSQuery import PBSQuery
420 p = PBSQuery()
421
422 jobs = p.getjobs()
423 for name, job in jobs.items()
424        print job
425
426=========== Version 2.7.4
427 New versions fixes an bug in function pbs.pbs_statnode().
428 Reported by: Keith Poirier
429
430 Configure support:
431 Contributed By: Yaroslav Halchenko
432
433 Debian package support:
434 Contributed By: Yaroslav Halchenko
435
436 Added Support for Debuging the interface, You must edit pbs_wrap.c
437 and search for SARA_DEBUG:
438        #define SARA_DEBUG 1
439 Implemented by: Ramon Bastiaans
440
441 examples/pbsmon.py:
442        - Fixed an error when regex fails to determine node number,
443          no status was displayed
444        - It will now display and 'j' if the node is free for the batch
445          system and a job runs on the node (SMP-systems).
446 Changed by: Bas van der Vlies
447
448 examples/rack_pbsmon.py:
449        - An pbsmon that display node info per rack
450        - Edit the rack_pbsmon.py to adjust the values. It requires
451          that hostname contains rack and node id's,
452          eg: gb-r<number>n<number>
453 Contributed by: Walter de Jong
454
455=========== Version 2.7.3
456 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
457 Resource and QUEing manager).
458
459 The interface support OPENPBS and TORQUE.
460
461=========== Version 2.7.2
462 Some minor changes to include files. Now the Scalable PBS keywords
463 are also supported ( server attribute names: node_ping_rate and
464 node_check_rate). This does not interfere with the openpbs software.
465
466 pbsmon.py could not handle an one node cluster. Fixed it.
467
468=========== Version 2.7
469 Forgot to wrap the pbs_statfree() function. So we could not free
470 allocated memory from functions that return 'struct batch_status *'
471 like pbs_statjob().
472
473 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
474 Daniel Olson
475
476=========== Version 2.6.1
477 Setup.py now checks if all openpbs libraries are installed to compile the
478 package.
479
480=========== Version 2.6
481
482 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
483 function. This bug prevented to pass 'struct attrl' variables to this
484 function. Thanks to Evelyn Shiu for reporting this bug.
485
486
487=========== Version 2.5
488 Added the OpenPbs logging functions (log.h and liblog.a). There
489 is an example in the examples directory: logpbs.py
490
491 Fixed a bug in examples/resmom_info.py used the default port for
492 pbs_resmon if getservbyname fails.
493
494 Fixed a bug in resmom code. If the user has no permissions to query
495 the pbs_mom daemon a empty string is returned. Now we check for
496 the empty string. So we do not get a python exception.
497
498 Fixed a bug in setup.py forgot the exit statement when we did not
499 find the pbs libraries.
500
501=========== Version 2.3
502 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
503
504 Removed some obsolete code.
505
506 As for now we make use of the distutils module. So the package
507 is automatically compiled and installed in proper directories, usage:
508    python setup.py install
509 This requires that the package 'distutils' must be installed. As for
510 version 1.6 and higher it is in the distribution.
511
512 Added a function that prints the pbs python interface version. So we
513 can check if we have the right interface version:
514 import pbs
515 print pbs.version()
516 
517
518=========== Version 2.2
519
520In this release we can query the pbs_mom daemon with the aid of
521resource management functions:
522   pbs.openrm()
523   pbs.closerm()
524   pbs.downrm()
525   pbs.configrm()
526   pbs.addreq()
527   pbs.allreq()
528   pbs.flushreq()
529   pbs.activereq()
530   pbs.fullresp()
531   pbs.getreq()
532
533There is also a python function:
534   pbs.get_mom_values(id, [list]):
535     id   - connection id of pbs.openrm(node-name, port-number)
536     list - Is a optional paramter. A list of resource keywords
537
538   If 'list' is not supplied then this function will get the values
539   for the standard resource keywords, eg uname, loadave, .. + 'arch'
540   depended keywords. The arch feature is only implemented for linux,
541   but it can easily be ported to other oses.
542
543   if 'list' is supplied then we only fetch values for the keywords
544   that are in the list.
545
546   The function returns a dictonary. The keys are the resource keywords.
547
548   See 'resmom_info.py' for a example of this new feature.
549
550=========== Version 2.0
551
552The previous version was a simple interface above the PBS C API LIB.  This
553interface has gone a major changes. The most noticeable change is  that
554the functions accept/return Python lists instead of C-lists.  So you can now
555use standard Python syntax for manipulating Python lists. If you are familar
556with the Python syntax then you will appreciate this new interface.
557
558NOTE:
559  This interface is NOT compatible with the old one
560
561Here are some guidelines to convert your code to the new one:
562
563The constructors for the struct has gone a major change:
564  old code:
565    temp = pbs.new_attrl();
566    attrl_p = pbs.attrlPtr(temp)
567    attrl_p.name = 'state'
568
569  new_code:
570     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
571     attr_l[0].name = 'state'
572
573
574The pbs_stat functions returns Python lists instead of C-lists. There is
575NO next field anymore:
576  old_code:
577    temp = pbs.new_attrl();
578    attrl_p = pbs.attrlPtr(temp)
579    attrl_p.this = 'NULL'
580
581    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
582
583    while nodes.this != 'NULL':
584      print nodes.name
585
586      node_attrl = nodes.attribs
587      while node_attrl.this != 'NULL':
588        print '\t', node_attrl.name, '=', node_attrl.value
589        node_attrl = node_attrl.next
590
591    nodes = nodes.next
592
593
594  new_code:
595    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
596    for node in nodes:
597      print node.name
598      for attrib in node.attribs:
599        print '\t', attrib.name, '=', attrib.value
600
601Another advantage is you can use the print statement to show the
602connect of attrl and attropl classes:
603   attr_l = pbs.new_attrl(2)
604   attr_l[0].name = 'bas'
605   attr_l[0].value = 'van der Vlies'
606
607   print  attr_l[0]
608   >> (bas,,van der Vlies)
609   
610I hope these examples illustrate the changes. If you specify a wrong type
611for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.