source: trunk/CHANGES @ 258

Last change on this file since 258 was 258, checked in by bas, 14 years ago

add author
  • Property svn:keywords set to Id
File size: 19.3 KB
Line 
1=========== 4.2.0
2 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
3  (jsbronder add gentoo dot org) for the information and
4  explanation.
5
6 * examples/pbsmon.py. Patched so it also works for torque 2.5
7   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
8   Applied by: Bas van der Vlies
9
10 * setup.py.in changed determine which torque version is installed
11   so we also support snapshots.
12   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
13   Applied by: Bas van der Vlies
14
15 * PBSQuery:
16   - get_nodes function for jobs now works for old and new datastructure and
17     gives correct results if unique=True
18     Reported by + idea for patch: Shunjie Lau (eijnuhs add gmail dot com)
19     Author: Bas van der Vlies
20
21 * Added AdvanceParser module. new_rack_pbsmon.py and sara_nodes.py make use of
22   this module
23   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
24
25 * examples/new_rack_pbsmon.py:
26     - Better specification for node expression
27     - added a new option -l/--list to list the state of a nodes and which job
28       is running on it, eg:
29             new_rack_pbsmon.py - l gb-r[1-3]n[3-6] gb-r4[n1-20]
30
31   Authors: Dennis Stam & Bas van der Vlies
32   Applied by: Bas van der Vlies
33
34 * examples/sara_nodes.py:
35   An utility to set/clear notes for multiple nodes, see ..
36   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
37
38 * setup.py.in:
39   Added support for runtime-libraries when torque is installed in a non-default
40   location, closes #22
41   Author: Mattias (slabanja at chalmers dot se)
42   Applied by: Bas van der Vlies
43
44 * Changed build system for debian
45   Authors: Dennis Stam & Bas van der Vlies
46
47=========== 4.1.2
48 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
49   structure.
50   Author: Bas van der Vlies
51
52 * PBSQuery:
53     - fixed and error in get_jobs function for nodes. Did not
54       support new data structure and uniq did not work.
55       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
56
57=========== 4.1.0
58 * Delete function pbs.version() replaced by
59    * pbs.version
60        This is a string variable
61    * pbs.version_info
62        Is a tuple variable. So we can use it in scripts
63
64   This analogue with sys module, closes #20
65
66   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
67   Author: Bas van der Vlies
68
69 * More robust make/configure/setup.py
70   Reported by:
71        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
72        - Shunjie Lau <eijnuhs add gmail dot com>
73   Author: Bas van der Vlies
74
75
76 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
77   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
78   Author: Bas van der Vlies
79   
80=========== 4.0.2
81  * Quick fix so we do not get any undefined symbols. Forgot to
82    add the torque library when we compile the package.
83    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
84    Author: Bas van der Vlies
85
86=========== 4.0.1
87  * Fixed a bug in setting the right library path  for compiling.
88    Reported by: Chad Vizino ( vizino add psc dot edu )
89    Author: Bas van der Vlies
90   
91=========== 4.0.0
92  * pbs.py added some new functions for 2.4 version:
93    - pbs_fbserver(void);
94    - pbs_get_server_list(void);
95    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
96    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
97    - pbs_checkpointjob(int connect, char *job_id, char *extend);
98    - log_ext(int,char *,char *,int);
99    - log_init(char *, char *);
100    - log_remove_old(char *,unsigned long);
101
102    Note: version 2.1.X is still supported
103
104  * autotools environment
105    - Added support for finding pbs-config (borrowed for mpiexec source)
106    - Detecting which version of torque is installed
107    - only support installations that have pbs-config installed.
108    - Removed old code for installations that do not a have pbs-config
109      installed.
110
111  * PBSQuery.py
112    - There was a bug in the new data interface (Resource_List for jobs)
113      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
114      Applied by with some minor changes: Bas van der Vlies
115
116    - The new data structure is the default. You can switch back to the old
117      one with:
118            p = PBSQuery()
119            p.old_data_structure()
120
121      Author: Bas van der Vlies
122
123   * examples/ha_server.py
124     An example how to use the High Availabilty torque functions
125     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
126
127=========== 3.6.0
128  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
129    Reported by: Dennis Stam
130    Fixed by: Bas van der Vlies
131
132=========== 3.5.0
133  * PBSQuery
134        The class functions of node, job and queue support old and new data
135        structure.
136
137        Changed the behaviour of the new data stucture, We can use it as
138        dictionary and as class attribute, this is equivalent, eg:
139          - print node['np'] and print node.np
140
141        for a node we parse the 'status' line and split on '=' char, You now can
142        use these statements, eg
143          - print node.status.arch     (node['status'].arch or node['status']['arch'])
144          - print node.status.nsession
145
146        for a job we parse the 'Variable_List' line and split on '=' char, You now can
147        use the statements, eg:
148          - print job.Variable_List.PBS_O_WORKDIR
149          - print job.Variable_List.PBS_O_HOME
150
151        For more info see examples/new_interface.py
152
153    Author: Bas van der Vlies
154
155 * new_rack_pbsmon.py
156        Rewrite to new data structure and automatically determine how many nodes
157        and racks cluster has and skip printing of empty racks (default), use -w/--wide
158        for old behaviour.
159
160   Author: Bas van der Vlies
161
162
163=========== 3.2.0
164        - PBSQuery:
165          New data structure. In the old structure it is a dictionary
166          with a value and the value is a string. This is changed
167          that dictionary values are now of type list or dictionary depends
168          on the value of the keyword, eg for a node:
169            - np = 2:
170              * node['np'] = [ '2' ] 
171
172            - properties = cores2, mem4gb, parallel
173              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
174
175            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
176              * node['status']['arch'] = [ 'x86_64' ]
177              * node['status']['sessions'] = [ '222599' ]
178              * ...
179
180         The data structure is activated by the function:
181          * new_data_structure()
182
183        In a future release it will be come the default.
184        example:
185                p = PBSQuery()
186                p.new_data_structure()
187
188                nodes = p.getnodes()
189                print nodes.np, nodes['np']
190
191        Author: Bas van der Vlies
192
193        - PBSQuery
194          For old and new data structure we now can use another syntax:
195           * node['np'] and node.np are equivalent
196
197          This new syntax works for all keywords.
198
199        Author: Bas van der Vlies
200
201        - Added iter object for job, node, queue and server objects, eg:
202          node = p.getnode('gb-r1n1')
203          print node.name
204          for attrib in node:
205                print '%\t%s = %s' %(attrib, node[attrib])
206        Author: Bas van der Vlies
207
208        - fixed an error in getnode, getqueue and getjob, return empty
209          dictionary if not found
210        Author: Bas van der Vlies
211       
212        - New build system for rpm packages, make -f Makefile.rpm
213          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
214
215        Applied: Bas van der Vlies
216
217=========== 3.0.1
218        - PBSQuery
219                * Removed a testing code line
220                Reported By: sam @ vpac dot org
221                Fixed by: Bas van der Vlies
222
223=========== 3.0.0
224        - PBSQuery
225                * Cleaned up some old lines of code which didn't do anything.
226                * Changed the returnvalue of getqueue, getnode and getjob. Now
227                  they return an instance of the object, instead of a dict.
228                  This is API change so we bumped the version to 3.0.0
229                * Now the getnode-function accepts the short name as well as
230                  the long name as an argument.
231
232                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
233
234        - pbs.py, _pbs.so, PBSQuery.py
235                * Fixed a memory leak when used in daemon mode
236                Author: Bas van der Vlies <basv@sara.nl>
237
238=========== 2.9.8
239        - pbs_python.spec file patch added libdir and python defines so it
240          will build for RHEL5.1/CentOS 5.1
241          by        : Michael Sternberg <sternberg add anl dot gov>
242          applied by: Bas van der Vlies
243
244        - Updated the header files for pbs_python. So it can use some new
245          defines variables/defines, eg:
246                * ATTR_NODE_status
247                * ATTR_NODE_note
248                * ...
249          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
250
251        - Two new functions to log.h:
252                log_roll(int size);
253                log_size(void);
254          Added by: Bas van der Vlies
255
256        - We now have proper destructors (free memory) in SWIG for attrl and
257          attropl attributes.
258          Added by: Bas van der Vlies
259
260        - PBSQuery
261                * Added getnodes_with_property() function. This will get all nodes with
262                  a certain property, eg:
263                   - get all nodes with property 'cores8' and return all attributes:
264                      getnodes_with_property('cores8')
265               
266                   - get all nodes with property 'cores2' and only return the
267                     attributes 'state' and 'properties':
268                      attrl = [ 'state', 'properties' ]
269                      getnodes_with_property('cores2', attrl)
270               
271                     Only for torque version > 2.1.0
272             
273                Added by: Bas van der Vlies
274
275                * Added to class node the function
276                 - get_jobs(self, unique=None)
277                   Returns a list of the currently running job-id('s) on the node.
278                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
279
280                * Added get_server_name() function.
281                  This will return the PBS-server's name.
282                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
283
284                * Added to class _PBSobject the function
285                  - uniq(self, list)
286                    Filters out unique items of a list.
287                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
288
289                * Added to class job the function
290                  - get_nodes(self, unique=None)
291                    Returns a list of the nodes which run the job.
292                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
293
294=========== Version 2.9.4
295        -  Function pbs.pbs_geterrmsg does not exits any more is
296           replaced by pbs.error():
297                errno, text = pbs.error()
298                print errno, text
299           Reported by: pk at q-leap dot com
300           Fixed by   : Bas van der Vlies
301
302        - PBSQuery.py
303           * Added get_version() for server class, returns the version of
304             the batch server
305             Added by    : Bas van der Vlies
306
307           * Added get_nodes() for job class, returns a list of nodes on
308             which the job is run
309             Added by    : Bas van der Vlies
310
311           * PBSQuery.py rewrite make use of UserDict module.
312             Suggested by: Ramon Bastiaans
313             Fixed by    : Bas van der Vlies
314
315=========== Version 2.9.2
316        - The function pbs_rescquery() did not work. Fixed it
317        - The function avail(), did not work fixed it
318        - The functions pbs_rescreserve and pbs_rescrelease did not work
319          fixed it
320
321        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
322          nodes was wrong and state down has topmost priority
323
324=========== Version 2.9.0
325        - Added support for new torque build system 2.1.0
326        - Added new_rack_pbsmon.py
327          Authors: Walter de Jong & Bas van der Vlies
328
329=========== Version 2.8.2
330
331Configure has a new option:
332        --with-pbsdir=PATH
333
334If set it will use PATH specified as argument to
335find the pbs/torque libraries. If unset it will use
336some useful defaults to find the libaries.
337Suggested by: Troy Baer
338
339PBSQuery.py:
340        - If we can not make a connection with the server
341          then raise an exception and let the programmer
342          decides what to do.
343
344pbs_python.spec:
345        - It can now handle mode bits
346          Author: Martin Pels from SARA
347       
348=========== Version 2.8.0
349
350setup.py:
351        - Added /usr/lib to PBS_LIB_DIR
352
353pbs.py:
354        - Added new function "pbs.error()". This function checks if
355          an error has occured with a pbs function, eg:
356            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
357            error_number, err_txt = pbs.error()
358            if error_number:
359                print error_number, err_txt
360
361          prints the following message if script A.tsk does not exists:
362            15042 (qsub) cannot access script file
363
364pbs_python.spec:
365        - to make rpm packages from the source package, initial version
366        Author: Martin Pels from SARA
367
368README:
369        - Now requires python 2.1 and higher
370        - Explain how to build DEBIAN package
371
372=========== Version 2.7.10
373
374PBSQuery.py:
375        Fixed an error. You always got all resources from eg nodes
376        even when you supplied an attribute list where you only
377        requested the 'state' of the node.
378
379        Updated the inline python documentation
380
381=========== Version 2.7.9
382
383Fixed an error the previous version was in DEBUG mode. So
384you get an lot undesired output on the screen.
385
386debian package now also generated a package with the write
387version info.
388
389=========== Version 2.7.8
390
391Setup.py:
392        Can also handle ROCKS installations + the version
393        info is now the same as pbs.py
394        Thanks to: Roy Dragseth
395
396=========== Version 2.7.7
397
398PBSQuery module:
399  It can now be used in daemon programs. The pbs_server closes
400  the connection after an certain amount of time. This is fixed,
401  before every query there is an new connection and if the query
402  is finished the connection is closed.
403 
404=========== Version 2.7.6
405
406Now setup.py can also handle old Oscar Installations.
407Thanks To: Robin Humble
408
409Fixed an print statement fix in new_interface.py:
410Thanks To: Robin Humble
411
412Made some improvements for PBSQuery module:
413        - Updated the documentation
414        - Added the has_key() function to all
415          PBSObjects. So that the behaviour is similar to
416          an dictionary
417          Suggested By: Ramon Bastiaans
418       
419=========== Version 2.7.5
420 Added PBSQuery module.  This module requires pbs.py and it a simple module
421 for querying the pbs server. The documentation is in the module. Use for
422 example ipython to read it.
423
424 Written By: Roy Dragseth
425             Bas van der Vlies
426
427 eg: ( see also examples/new_interface.py)
428
429 from PBSQuery import PBSQuery
430 p = PBSQuery()
431
432 jobs = p.getjobs()
433 for name, job in jobs.items()
434        print job
435
436=========== Version 2.7.4
437 New versions fixes an bug in function pbs.pbs_statnode().
438 Reported by: Keith Poirier
439
440 Configure support:
441 Contributed By: Yaroslav Halchenko
442
443 Debian package support:
444 Contributed By: Yaroslav Halchenko
445
446 Added Support for Debuging the interface, You must edit pbs_wrap.c
447 and search for SARA_DEBUG:
448        #define SARA_DEBUG 1
449 Implemented by: Ramon Bastiaans
450
451 examples/pbsmon.py:
452        - Fixed an error when regex fails to determine node number,
453          no status was displayed
454        - It will now display and 'j' if the node is free for the batch
455          system and a job runs on the node (SMP-systems).
456 Changed by: Bas van der Vlies
457
458 examples/rack_pbsmon.py:
459        - An pbsmon that display node info per rack
460        - Edit the rack_pbsmon.py to adjust the values. It requires
461          that hostname contains rack and node id's,
462          eg: gb-r<number>n<number>
463 Contributed by: Walter de Jong
464
465=========== Version 2.7.3
466 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
467 Resource and QUEing manager).
468
469 The interface support OPENPBS and TORQUE.
470
471=========== Version 2.7.2
472 Some minor changes to include files. Now the Scalable PBS keywords
473 are also supported ( server attribute names: node_ping_rate and
474 node_check_rate). This does not interfere with the openpbs software.
475
476 pbsmon.py could not handle an one node cluster. Fixed it.
477
478=========== Version 2.7
479 Forgot to wrap the pbs_statfree() function. So we could not free
480 allocated memory from functions that return 'struct batch_status *'
481 like pbs_statjob().
482
483 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
484 Daniel Olson
485
486=========== Version 2.6.1
487 Setup.py now checks if all openpbs libraries are installed to compile the
488 package.
489
490=========== Version 2.6
491
492 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
493 function. This bug prevented to pass 'struct attrl' variables to this
494 function. Thanks to Evelyn Shiu for reporting this bug.
495
496
497=========== Version 2.5
498 Added the OpenPbs logging functions (log.h and liblog.a). There
499 is an example in the examples directory: logpbs.py
500
501 Fixed a bug in examples/resmom_info.py used the default port for
502 pbs_resmon if getservbyname fails.
503
504 Fixed a bug in resmom code. If the user has no permissions to query
505 the pbs_mom daemon a empty string is returned. Now we check for
506 the empty string. So we do not get a python exception.
507
508 Fixed a bug in setup.py forgot the exit statement when we did not
509 find the pbs libraries.
510
511=========== Version 2.3
512 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
513
514 Removed some obsolete code.
515
516 As for now we make use of the distutils module. So the package
517 is automatically compiled and installed in proper directories, usage:
518    python setup.py install
519 This requires that the package 'distutils' must be installed. As for
520 version 1.6 and higher it is in the distribution.
521
522 Added a function that prints the pbs python interface version. So we
523 can check if we have the right interface version:
524 import pbs
525 print pbs.version()
526 
527
528=========== Version 2.2
529
530In this release we can query the pbs_mom daemon with the aid of
531resource management functions:
532   pbs.openrm()
533   pbs.closerm()
534   pbs.downrm()
535   pbs.configrm()
536   pbs.addreq()
537   pbs.allreq()
538   pbs.flushreq()
539   pbs.activereq()
540   pbs.fullresp()
541   pbs.getreq()
542
543There is also a python function:
544   pbs.get_mom_values(id, [list]):
545     id   - connection id of pbs.openrm(node-name, port-number)
546     list - Is a optional paramter. A list of resource keywords
547
548   If 'list' is not supplied then this function will get the values
549   for the standard resource keywords, eg uname, loadave, .. + 'arch'
550   depended keywords. The arch feature is only implemented for linux,
551   but it can easily be ported to other oses.
552
553   if 'list' is supplied then we only fetch values for the keywords
554   that are in the list.
555
556   The function returns a dictonary. The keys are the resource keywords.
557
558   See 'resmom_info.py' for a example of this new feature.
559
560=========== Version 2.0
561
562The previous version was a simple interface above the PBS C API LIB.  This
563interface has gone a major changes. The most noticeable change is  that
564the functions accept/return Python lists instead of C-lists.  So you can now
565use standard Python syntax for manipulating Python lists. If you are familar
566with the Python syntax then you will appreciate this new interface.
567
568NOTE:
569  This interface is NOT compatible with the old one
570
571Here are some guidelines to convert your code to the new one:
572
573The constructors for the struct has gone a major change:
574  old code:
575    temp = pbs.new_attrl();
576    attrl_p = pbs.attrlPtr(temp)
577    attrl_p.name = 'state'
578
579  new_code:
580     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
581     attr_l[0].name = 'state'
582
583
584The pbs_stat functions returns Python lists instead of C-lists. There is
585NO next field anymore:
586  old_code:
587    temp = pbs.new_attrl();
588    attrl_p = pbs.attrlPtr(temp)
589    attrl_p.this = 'NULL'
590
591    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
592
593    while nodes.this != 'NULL':
594      print nodes.name
595
596      node_attrl = nodes.attribs
597      while node_attrl.this != 'NULL':
598        print '\t', node_attrl.name, '=', node_attrl.value
599        node_attrl = node_attrl.next
600
601    nodes = nodes.next
602
603
604  new_code:
605    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
606    for node in nodes:
607      print node.name
608      for attrib in node.attribs:
609        print '\t', attrib.name, '=', attrib.value
610
611Another advantage is you can use the print statement to show the
612connect of attrl and attropl classes:
613   attr_l = pbs.new_attrl(2)
614   attr_l[0].name = 'bas'
615   attr_l[0].value = 'van der Vlies'
616
617   print  attr_l[0]
618   >> (bas,,van der Vlies)
619   
620I hope these examples illustrate the changes. If you specify a wrong type
621for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.