source: trunk/CHANGES @ 269

Last change on this file since 269 was 269, checked in by dennis, 13 years ago

Fixed AdvancedParser? when using range 01-12

  • Property svn:keywords set to Id
File size: 20.5 KB
Line 
1=========== 4.X.X
2 * New generated files for pbs_wrap.c and pbs.py to support python 3.X versions
3 * Fixed AdvancedParser when using 01-12 range the zero was not appended.
4   Reported by: Ramon Bastiaans
5   Author: Dennis Stam
6
7=========== 4.3.0
8 * new_rack_pbsmon.py: For backward compatibility added -a/--all option.
9   Has the same functionality as -s/--summary
10   Author: Bas van der Vlies
11
12 * Added some license info to debian package and include GPL and LGPL 3.0
13   files.
14   Suggested by: W. Trevor King <wking at drexel dot edu>
15
16 * PBSQuery.py: Added support for new feature in node status field:
17    * message=EVENT:<key=value:>+
18    * message=ERROR <text>
19
20  Added the following syntax for node:
21        * node['error']
22        * node['event']
23
24  eg: node: n1
25    state = free
26    np = 8
27    properties = ib,switch1,highmem
28    ntype = cluster
29    jobs = 0/567403.sara.nl, 1/567403.sara.nl
30    status = ..,loadave=0.00,message=EVENT:sample.time=1288864220.003:cputotals.user=0:iconnect.pktout=0,netload=3487600394,state=free,...
31
32  example code:
33        p = PBSQuery()
34        node = p.getnode(n1)
35        print node.event['sample.time']
36        print node['event']['sample.time']
37
38   Requested by: Roy Dragseth <roy dot dragseth add uit dotno>
39   Author: Bas van der Vlies
40
41=========== 4.2.0
42 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
43   (jsbronder add gentoo dot org) for the information and
44   explanation.
45
46 * examples/pbsmon.py. Patched so it also works for torque 2.5
47   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
48   Applied by: Bas van der Vlies
49
50 * setup.py.in changed determine which torque version is installed
51   so we also support snapshots.
52   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
53   Applied by: Bas van der Vlies
54
55 * PBSQuery:
56   - get_nodes function for jobs now works for old and new datastructure and
57     gives correct results if unique=True
58     Reported by + idea for patch: Shunjie Lau (eijnuhs add gmail dot com)
59     Author: Bas van der Vlies
60
61 * Added AdvanceParser module. new_rack_pbsmon.py and sara_nodes.py make use of
62   this module
63   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
64
65 * examples/new_rack_pbsmon.py:
66     - Better specification for node expression
67     - added a new option -l/--list to list the state of a nodes and which job
68       is running on it, eg:
69             new_rack_pbsmon.py - l gb-r[1-3]n[3-6] gb-r4[n1-20]
70
71   Authors: Dennis Stam & Bas van der Vlies
72   Applied by: Bas van der Vlies
73
74 * examples/sara_nodes.py:
75   An utility to set/clear notes for multiple nodes, see ..
76   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
77
78 * setup.py.in:
79   Added support for runtime-libraries when torque is installed in a non-default
80   location, closes #22
81   Author: Mattias (slabanja at chalmers dot se)
82   Applied by: Bas van der Vlies
83
84 * Changed build system for debian
85   Authors: Dennis Stam & Bas van der Vlies
86
87=========== 4.1.2
88 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
89   structure.
90   Author: Bas van der Vlies
91
92 * PBSQuery:
93     - fixed and error in get_jobs function for nodes. Did not
94       support new data structure and uniq did not work.
95       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
96
97=========== 4.1.0
98 * Delete function pbs.version() replaced by
99    * pbs.version
100        This is a string variable
101    * pbs.version_info
102        Is a tuple variable. So we can use it in scripts
103
104   This analogue with sys module, closes #20
105
106   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
107   Author: Bas van der Vlies
108
109 * More robust make/configure/setup.py
110   Reported by:
111        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
112        - Shunjie Lau <eijnuhs add gmail dot com>
113   Author: Bas van der Vlies
114
115
116 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
117   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
118   Author: Bas van der Vlies
119   
120=========== 4.0.2
121  * Quick fix so we do not get any undefined symbols. Forgot to
122    add the torque library when we compile the package.
123    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
124    Author: Bas van der Vlies
125
126=========== 4.0.1
127  * Fixed a bug in setting the right library path  for compiling.
128    Reported by: Chad Vizino ( vizino add psc dot edu )
129    Author: Bas van der Vlies
130   
131=========== 4.0.0
132  * pbs.py added some new functions for 2.4 version:
133    - pbs_fbserver(void);
134    - pbs_get_server_list(void);
135    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
136    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
137    - pbs_checkpointjob(int connect, char *job_id, char *extend);
138    - log_ext(int,char *,char *,int);
139    - log_init(char *, char *);
140    - log_remove_old(char *,unsigned long);
141
142    Note: version 2.1.X is still supported
143
144  * autotools environment
145    - Added support for finding pbs-config (borrowed for mpiexec source)
146    - Detecting which version of torque is installed
147    - only support installations that have pbs-config installed.
148    - Removed old code for installations that do not a have pbs-config
149      installed.
150
151  * PBSQuery.py
152    - There was a bug in the new data interface (Resource_List for jobs)
153      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
154      Applied by with some minor changes: Bas van der Vlies
155
156    - The new data structure is the default. You can switch back to the old
157      one with:
158            p = PBSQuery()
159            p.old_data_structure()
160
161      Author: Bas van der Vlies
162
163   * examples/ha_server.py
164     An example how to use the High Availabilty torque functions
165     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
166
167=========== 3.6.0
168  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
169    Reported by: Dennis Stam
170    Fixed by: Bas van der Vlies
171
172=========== 3.5.0
173  * PBSQuery
174        The class functions of node, job and queue support old and new data
175        structure.
176
177        Changed the behaviour of the new data stucture, We can use it as
178        dictionary and as class attribute, this is equivalent, eg:
179          - print node['np'] and print node.np
180
181        for a node we parse the 'status' line and split on '=' char, You now can
182        use these statements, eg
183          - print node.status.arch     (node['status'].arch or node['status']['arch'])
184          - print node.status.nsession
185
186        for a job we parse the 'Variable_List' line and split on '=' char, You now can
187        use the statements, eg:
188          - print job.Variable_List.PBS_O_WORKDIR
189          - print job.Variable_List.PBS_O_HOME
190
191        For more info see examples/new_interface.py
192
193    Author: Bas van der Vlies
194
195 * new_rack_pbsmon.py
196        Rewrite to new data structure and automatically determine how many nodes
197        and racks cluster has and skip printing of empty racks (default), use -w/--wide
198        for old behaviour.
199
200   Author: Bas van der Vlies
201
202
203=========== 3.2.0
204        - PBSQuery:
205          New data structure. In the old structure it is a dictionary
206          with a value and the value is a string. This is changed
207          that dictionary values are now of type list or dictionary depends
208          on the value of the keyword, eg for a node:
209            - np = 2:
210              * node['np'] = [ '2' ] 
211
212            - properties = cores2, mem4gb, parallel
213              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
214
215            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
216              * node['status']['arch'] = [ 'x86_64' ]
217              * node['status']['sessions'] = [ '222599' ]
218              * ...
219
220         The data structure is activated by the function:
221          * new_data_structure()
222
223        In a future release it will be come the default.
224        example:
225                p = PBSQuery()
226                p.new_data_structure()
227
228                nodes = p.getnodes()
229                print nodes.np, nodes['np']
230
231        Author: Bas van der Vlies
232
233        - PBSQuery
234          For old and new data structure we now can use another syntax:
235           * node['np'] and node.np are equivalent
236
237          This new syntax works for all keywords.
238
239        Author: Bas van der Vlies
240
241        - Added iter object for job, node, queue and server objects, eg:
242          node = p.getnode('gb-r1n1')
243          print node.name
244          for attrib in node:
245                print '%\t%s = %s' %(attrib, node[attrib])
246        Author: Bas van der Vlies
247
248        - fixed an error in getnode, getqueue and getjob, return empty
249          dictionary if not found
250        Author: Bas van der Vlies
251       
252        - New build system for rpm packages, make -f Makefile.rpm
253          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
254
255        Applied: Bas van der Vlies
256
257=========== 3.0.1
258        - PBSQuery
259                * Removed a testing code line
260                Reported By: sam @ vpac dot org
261                Fixed by: Bas van der Vlies
262
263=========== 3.0.0
264        - PBSQuery
265                * Cleaned up some old lines of code which didn't do anything.
266                * Changed the returnvalue of getqueue, getnode and getjob. Now
267                  they return an instance of the object, instead of a dict.
268                  This is API change so we bumped the version to 3.0.0
269                * Now the getnode-function accepts the short name as well as
270                  the long name as an argument.
271
272                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
273
274        - pbs.py, _pbs.so, PBSQuery.py
275                * Fixed a memory leak when used in daemon mode
276                Author: Bas van der Vlies <basv@sara.nl>
277
278=========== 2.9.8
279        - pbs_python.spec file patch added libdir and python defines so it
280          will build for RHEL5.1/CentOS 5.1
281          by        : Michael Sternberg <sternberg add anl dot gov>
282          applied by: Bas van der Vlies
283
284        - Updated the header files for pbs_python. So it can use some new
285          defines variables/defines, eg:
286                * ATTR_NODE_status
287                * ATTR_NODE_note
288                * ...
289          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
290
291        - Two new functions to log.h:
292                log_roll(int size);
293                log_size(void);
294          Added by: Bas van der Vlies
295
296        - We now have proper destructors (free memory) in SWIG for attrl and
297          attropl attributes.
298          Added by: Bas van der Vlies
299
300        - PBSQuery
301                * Added getnodes_with_property() function. This will get all nodes with
302                  a certain property, eg:
303                   - get all nodes with property 'cores8' and return all attributes:
304                      getnodes_with_property('cores8')
305               
306                   - get all nodes with property 'cores2' and only return the
307                     attributes 'state' and 'properties':
308                      attrl = [ 'state', 'properties' ]
309                      getnodes_with_property('cores2', attrl)
310               
311                     Only for torque version > 2.1.0
312             
313                Added by: Bas van der Vlies
314
315                * Added to class node the function
316                 - get_jobs(self, unique=None)
317                   Returns a list of the currently running job-id('s) on the node.
318                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
319
320                * Added get_server_name() function.
321                  This will return the PBS-server's name.
322                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
323
324                * Added to class _PBSobject the function
325                  - uniq(self, list)
326                    Filters out unique items of a list.
327                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
328
329                * Added to class job the function
330                  - get_nodes(self, unique=None)
331                    Returns a list of the nodes which run the job.
332                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
333
334=========== Version 2.9.4
335        -  Function pbs.pbs_geterrmsg does not exits any more is
336           replaced by pbs.error():
337                errno, text = pbs.error()
338                print errno, text
339           Reported by: pk at q-leap dot com
340           Fixed by   : Bas van der Vlies
341
342        - PBSQuery.py
343           * Added get_version() for server class, returns the version of
344             the batch server
345             Added by    : Bas van der Vlies
346
347           * Added get_nodes() for job class, returns a list of nodes on
348             which the job is run
349             Added by    : Bas van der Vlies
350
351           * PBSQuery.py rewrite make use of UserDict module.
352             Suggested by: Ramon Bastiaans
353             Fixed by    : Bas van der Vlies
354
355=========== Version 2.9.2
356        - The function pbs_rescquery() did not work. Fixed it
357        - The function avail(), did not work fixed it
358        - The functions pbs_rescreserve and pbs_rescrelease did not work
359          fixed it
360
361        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
362          nodes was wrong and state down has topmost priority
363
364=========== Version 2.9.0
365        - Added support for new torque build system 2.1.0
366        - Added new_rack_pbsmon.py
367          Authors: Walter de Jong & Bas van der Vlies
368
369=========== Version 2.8.2
370
371Configure has a new option:
372        --with-pbsdir=PATH
373
374If set it will use PATH specified as argument to
375find the pbs/torque libraries. If unset it will use
376some useful defaults to find the libaries.
377Suggested by: Troy Baer
378
379PBSQuery.py:
380        - If we can not make a connection with the server
381          then raise an exception and let the programmer
382          decides what to do.
383
384pbs_python.spec:
385        - It can now handle mode bits
386          Author: Martin Pels from SARA
387       
388=========== Version 2.8.0
389
390setup.py:
391        - Added /usr/lib to PBS_LIB_DIR
392
393pbs.py:
394        - Added new function "pbs.error()". This function checks if
395          an error has occured with a pbs function, eg:
396            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
397            error_number, err_txt = pbs.error()
398            if error_number:
399                print error_number, err_txt
400
401          prints the following message if script A.tsk does not exists:
402            15042 (qsub) cannot access script file
403
404pbs_python.spec:
405        - to make rpm packages from the source package, initial version
406        Author: Martin Pels from SARA
407
408README:
409        - Now requires python 2.1 and higher
410        - Explain how to build DEBIAN package
411
412=========== Version 2.7.10
413
414PBSQuery.py:
415        Fixed an error. You always got all resources from eg nodes
416        even when you supplied an attribute list where you only
417        requested the 'state' of the node.
418
419        Updated the inline python documentation
420
421=========== Version 2.7.9
422
423Fixed an error the previous version was in DEBUG mode. So
424you get an lot undesired output on the screen.
425
426debian package now also generated a package with the write
427version info.
428
429=========== Version 2.7.8
430
431Setup.py:
432        Can also handle ROCKS installations + the version
433        info is now the same as pbs.py
434        Thanks to: Roy Dragseth
435
436=========== Version 2.7.7
437
438PBSQuery module:
439  It can now be used in daemon programs. The pbs_server closes
440  the connection after an certain amount of time. This is fixed,
441  before every query there is an new connection and if the query
442  is finished the connection is closed.
443 
444=========== Version 2.7.6
445
446Now setup.py can also handle old Oscar Installations.
447Thanks To: Robin Humble
448
449Fixed an print statement fix in new_interface.py:
450Thanks To: Robin Humble
451
452Made some improvements for PBSQuery module:
453        - Updated the documentation
454        - Added the has_key() function to all
455          PBSObjects. So that the behaviour is similar to
456          an dictionary
457          Suggested By: Ramon Bastiaans
458       
459=========== Version 2.7.5
460 Added PBSQuery module.  This module requires pbs.py and it a simple module
461 for querying the pbs server. The documentation is in the module. Use for
462 example ipython to read it.
463
464 Written By: Roy Dragseth
465             Bas van der Vlies
466
467 eg: ( see also examples/new_interface.py)
468
469 from PBSQuery import PBSQuery
470 p = PBSQuery()
471
472 jobs = p.getjobs()
473 for name, job in jobs.items()
474        print job
475
476=========== Version 2.7.4
477 New versions fixes an bug in function pbs.pbs_statnode().
478 Reported by: Keith Poirier
479
480 Configure support:
481 Contributed By: Yaroslav Halchenko
482
483 Debian package support:
484 Contributed By: Yaroslav Halchenko
485
486 Added Support for Debuging the interface, You must edit pbs_wrap.c
487 and search for SARA_DEBUG:
488        #define SARA_DEBUG 1
489 Implemented by: Ramon Bastiaans
490
491 examples/pbsmon.py:
492        - Fixed an error when regex fails to determine node number,
493          no status was displayed
494        - It will now display and 'j' if the node is free for the batch
495          system and a job runs on the node (SMP-systems).
496 Changed by: Bas van der Vlies
497
498 examples/rack_pbsmon.py:
499        - An pbsmon that display node info per rack
500        - Edit the rack_pbsmon.py to adjust the values. It requires
501          that hostname contains rack and node id's,
502          eg: gb-r<number>n<number>
503 Contributed by: Walter de Jong
504
505=========== Version 2.7.3
506 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
507 Resource and QUEing manager).
508
509 The interface support OPENPBS and TORQUE.
510
511=========== Version 2.7.2
512 Some minor changes to include files. Now the Scalable PBS keywords
513 are also supported ( server attribute names: node_ping_rate and
514 node_check_rate). This does not interfere with the openpbs software.
515
516 pbsmon.py could not handle an one node cluster. Fixed it.
517
518=========== Version 2.7
519 Forgot to wrap the pbs_statfree() function. So we could not free
520 allocated memory from functions that return 'struct batch_status *'
521 like pbs_statjob().
522
523 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
524 Daniel Olson
525
526=========== Version 2.6.1
527 Setup.py now checks if all openpbs libraries are installed to compile the
528 package.
529
530=========== Version 2.6
531
532 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
533 function. This bug prevented to pass 'struct attrl' variables to this
534 function. Thanks to Evelyn Shiu for reporting this bug.
535
536
537=========== Version 2.5
538 Added the OpenPbs logging functions (log.h and liblog.a). There
539 is an example in the examples directory: logpbs.py
540
541 Fixed a bug in examples/resmom_info.py used the default port for
542 pbs_resmon if getservbyname fails.
543
544 Fixed a bug in resmom code. If the user has no permissions to query
545 the pbs_mom daemon a empty string is returned. Now we check for
546 the empty string. So we do not get a python exception.
547
548 Fixed a bug in setup.py forgot the exit statement when we did not
549 find the pbs libraries.
550
551=========== Version 2.3
552 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
553
554 Removed some obsolete code.
555
556 As for now we make use of the distutils module. So the package
557 is automatically compiled and installed in proper directories, usage:
558    python setup.py install
559 This requires that the package 'distutils' must be installed. As for
560 version 1.6 and higher it is in the distribution.
561
562 Added a function that prints the pbs python interface version. So we
563 can check if we have the right interface version:
564 import pbs
565 print pbs.version()
566 
567
568=========== Version 2.2
569
570In this release we can query the pbs_mom daemon with the aid of
571resource management functions:
572   pbs.openrm()
573   pbs.closerm()
574   pbs.downrm()
575   pbs.configrm()
576   pbs.addreq()
577   pbs.allreq()
578   pbs.flushreq()
579   pbs.activereq()
580   pbs.fullresp()
581   pbs.getreq()
582
583There is also a python function:
584   pbs.get_mom_values(id, [list]):
585     id   - connection id of pbs.openrm(node-name, port-number)
586     list - Is a optional paramter. A list of resource keywords
587
588   If 'list' is not supplied then this function will get the values
589   for the standard resource keywords, eg uname, loadave, .. + 'arch'
590   depended keywords. The arch feature is only implemented for linux,
591   but it can easily be ported to other oses.
592
593   if 'list' is supplied then we only fetch values for the keywords
594   that are in the list.
595
596   The function returns a dictonary. The keys are the resource keywords.
597
598   See 'resmom_info.py' for a example of this new feature.
599
600=========== Version 2.0
601
602The previous version was a simple interface above the PBS C API LIB.  This
603interface has gone a major changes. The most noticeable change is  that
604the functions accept/return Python lists instead of C-lists.  So you can now
605use standard Python syntax for manipulating Python lists. If you are familar
606with the Python syntax then you will appreciate this new interface.
607
608NOTE:
609  This interface is NOT compatible with the old one
610
611Here are some guidelines to convert your code to the new one:
612
613The constructors for the struct has gone a major change:
614  old code:
615    temp = pbs.new_attrl();
616    attrl_p = pbs.attrlPtr(temp)
617    attrl_p.name = 'state'
618
619  new_code:
620     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
621     attr_l[0].name = 'state'
622
623
624The pbs_stat functions returns Python lists instead of C-lists. There is
625NO next field anymore:
626  old_code:
627    temp = pbs.new_attrl();
628    attrl_p = pbs.attrlPtr(temp)
629    attrl_p.this = 'NULL'
630
631    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
632
633    while nodes.this != 'NULL':
634      print nodes.name
635
636      node_attrl = nodes.attribs
637      while node_attrl.this != 'NULL':
638        print '\t', node_attrl.name, '=', node_attrl.value
639        node_attrl = node_attrl.next
640
641    nodes = nodes.next
642
643
644  new_code:
645    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
646    for node in nodes:
647      print node.name
648      for attrib in node.attribs:
649        print '\t', attrib.name, '=', attrib.value
650
651Another advantage is you can use the print statement to show the
652connect of attrl and attropl classes:
653   attr_l = pbs.new_attrl(2)
654   attr_l[0].name = 'bas'
655   attr_l[0].value = 'van der Vlies'
656
657   print  attr_l[0]
658   >> (bas,,van der Vlies)
659   
660I hope these examples illustrate the changes. If you specify a wrong type
661for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.