source: trunk/CHANGES @ 271

Last change on this file since 271 was 271, checked in by bas, 12 years ago

Changed debian dependencies. closes #30
and fixed some conflicts with SARA packages

  • Property svn:keywords set to Id
File size: 21.1 KB
Line 
1=========== 4.3.3
2 * New generated files for pbs_wrap.c and pbs.py to support python 3.X versions
3   Author: Bas van der Vlies
4
5 * Fixed AdvancedParser when using 01-12 range the zero was not appended.
6   Reported by: Ramon Bastiaans
7   Author: Dennis Stam
8
9 * examples/sara_nodes.py: Catch the PBSQuery error if we can not make a
10   connection with the batch server and exit the program.
11   Author: Bas van der Vlies
12
13 * Remove the debian dependency on AdvancedParser. We now have
14   our own PBSAdvanceParser so it does not conflict with other SARA
15   packages.
16   Reported by: Ramon Bastiaans (SARA)
17   Fixed by: Bas van der Vlies
18
19 * Make pbs_torque only dependend on libtorque instead of torque and adjusted
20   the maintainer of the package, closes #30
21   Reported by: Guillermo Marcus  <guillermo dot marcus add@gmail dot com>
22   Fixed by: Bas van der Vlies
23   
24=========== 4.3.0
25 * new_rack_pbsmon.py: For backward compatibility added -a/--all option.
26   Has the same functionality as -s/--summary
27   Author: Bas van der Vlies
28
29 * Added some license info to debian package and include GPL and LGPL 3.0
30   files.
31   Suggested by: W. Trevor King <wking at drexel dot edu>
32
33 * PBSQuery.py: Added support for new feature in node status field:
34    * message=EVENT:<key=value:>+
35    * message=ERROR <text>
36
37  Added the following syntax for node:
38        * node['error']
39    * node['event']
40
41  eg: node: n1
42    state = free
43    np = 8
44    properties = ib,switch1,highmem
45    ntype = cluster
46    jobs = 0/567403.sara.nl, 1/567403.sara.nl
47    status = ..,loadave=0.00,message=EVENT:sample.time=1288864220.003:cputotals.user=0:iconnect.pktout=0,netload=3487600394,state=free,...
48
49  example code:
50        p = PBSQuery()
51    node = p.getnode(n1)
52    print node.event['sample.time']
53    print node['event']['sample.time']
54
55   Requested by: Roy Dragseth <roy dot dragseth add uit dotno>
56   Author: Bas van der Vlies
57
58=========== 4.2.0
59 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
60   (jsbronder add gentoo dot org) for the information and
61   explanation.
62
63 * examples/pbsmon.py. Patched so it also works for torque 2.5
64   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
65   Applied by: Bas van der Vlies
66
67 * setup.py.in changed determine which torque version is installed
68   so we also support snapshots.
69   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
70   Applied by: Bas van der Vlies
71
72 * PBSQuery:
73   - get_nodes function for jobs now works for old and new datastructure and
74     gives correct results if unique=True
75     Reported by + idea for patch: Shunjie Lau (eijnuhs add gmail dot com)
76     Author: Bas van der Vlies
77
78 * Added AdvanceParser module. new_rack_pbsmon.py and sara_nodes.py make use of
79   this module
80   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
81
82 * examples/new_rack_pbsmon.py:
83     - Better specification for node expression
84     - added a new option -l/--list to list the state of a nodes and which job
85       is running on it, eg:
86             new_rack_pbsmon.py - l gb-r[1-3]n[3-6] gb-r4[n1-20]
87
88   Authors: Dennis Stam & Bas van der Vlies
89   Applied by: Bas van der Vlies
90
91 * examples/sara_nodes.py:
92   An utility to set/clear notes for multiple nodes, see ..
93   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
94
95 * setup.py.in:
96   Added support for runtime-libraries when torque is installed in a non-default
97   location, closes #22
98   Author: Mattias (slabanja at chalmers dot se)
99   Applied by: Bas van der Vlies
100
101 * Changed build system for debian
102   Authors: Dennis Stam & Bas van der Vlies
103
104=========== 4.1.2
105 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
106   structure.
107   Author: Bas van der Vlies
108
109 * PBSQuery:
110     - fixed and error in get_jobs function for nodes. Did not
111       support new data structure and uniq did not work.
112       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
113
114=========== 4.1.0
115 * Delete function pbs.version() replaced by
116    * pbs.version
117        This is a string variable
118    * pbs.version_info
119        Is a tuple variable. So we can use it in scripts
120
121   This analogue with sys module, closes #20
122
123   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
124   Author: Bas van der Vlies
125
126 * More robust make/configure/setup.py
127   Reported by:
128        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
129        - Shunjie Lau <eijnuhs add gmail dot com>
130   Author: Bas van der Vlies
131
132
133 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
134   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
135   Author: Bas van der Vlies
136   
137=========== 4.0.2
138  * Quick fix so we do not get any undefined symbols. Forgot to
139    add the torque library when we compile the package.
140    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
141    Author: Bas van der Vlies
142
143=========== 4.0.1
144  * Fixed a bug in setting the right library path  for compiling.
145    Reported by: Chad Vizino ( vizino add psc dot edu )
146    Author: Bas van der Vlies
147   
148=========== 4.0.0
149  * pbs.py added some new functions for 2.4 version:
150    - pbs_fbserver(void);
151    - pbs_get_server_list(void);
152    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
153    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
154    - pbs_checkpointjob(int connect, char *job_id, char *extend);
155    - log_ext(int,char *,char *,int);
156    - log_init(char *, char *);
157    - log_remove_old(char *,unsigned long);
158
159    Note: version 2.1.X is still supported
160
161  * autotools environment
162    - Added support for finding pbs-config (borrowed for mpiexec source)
163    - Detecting which version of torque is installed
164    - only support installations that have pbs-config installed.
165    - Removed old code for installations that do not a have pbs-config
166      installed.
167
168  * PBSQuery.py
169    - There was a bug in the new data interface (Resource_List for jobs)
170      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
171      Applied by with some minor changes: Bas van der Vlies
172
173    - The new data structure is the default. You can switch back to the old
174      one with:
175            p = PBSQuery()
176            p.old_data_structure()
177
178      Author: Bas van der Vlies
179
180   * examples/ha_server.py
181     An example how to use the High Availabilty torque functions
182     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
183
184=========== 3.6.0
185  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
186    Reported by: Dennis Stam
187    Fixed by: Bas van der Vlies
188
189=========== 3.5.0
190  * PBSQuery
191        The class functions of node, job and queue support old and new data
192        structure.
193
194        Changed the behaviour of the new data stucture, We can use it as
195        dictionary and as class attribute, this is equivalent, eg:
196          - print node['np'] and print node.np
197
198        for a node we parse the 'status' line and split on '=' char, You now can
199        use these statements, eg
200          - print node.status.arch     (node['status'].arch or node['status']['arch'])
201          - print node.status.nsession
202
203        for a job we parse the 'Variable_List' line and split on '=' char, You now can
204        use the statements, eg:
205          - print job.Variable_List.PBS_O_WORKDIR
206          - print job.Variable_List.PBS_O_HOME
207
208        For more info see examples/new_interface.py
209
210    Author: Bas van der Vlies
211
212 * new_rack_pbsmon.py
213        Rewrite to new data structure and automatically determine how many nodes
214        and racks cluster has and skip printing of empty racks (default), use -w/--wide
215        for old behaviour.
216
217   Author: Bas van der Vlies
218
219
220=========== 3.2.0
221        - PBSQuery:
222          New data structure. In the old structure it is a dictionary
223          with a value and the value is a string. This is changed
224          that dictionary values are now of type list or dictionary depends
225          on the value of the keyword, eg for a node:
226            - np = 2:
227              * node['np'] = [ '2' ] 
228
229            - properties = cores2, mem4gb, parallel
230              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
231
232            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
233              * node['status']['arch'] = [ 'x86_64' ]
234              * node['status']['sessions'] = [ '222599' ]
235              * ...
236
237         The data structure is activated by the function:
238          * new_data_structure()
239
240        In a future release it will be come the default.
241        example:
242                p = PBSQuery()
243                p.new_data_structure()
244
245                nodes = p.getnodes()
246                print nodes.np, nodes['np']
247
248        Author: Bas van der Vlies
249
250        - PBSQuery
251          For old and new data structure we now can use another syntax:
252           * node['np'] and node.np are equivalent
253
254          This new syntax works for all keywords.
255
256        Author: Bas van der Vlies
257
258        - Added iter object for job, node, queue and server objects, eg:
259          node = p.getnode('gb-r1n1')
260          print node.name
261          for attrib in node:
262                print '%\t%s = %s' %(attrib, node[attrib])
263        Author: Bas van der Vlies
264
265        - fixed an error in getnode, getqueue and getjob, return empty
266          dictionary if not found
267        Author: Bas van der Vlies
268       
269        - New build system for rpm packages, make -f Makefile.rpm
270          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
271
272        Applied: Bas van der Vlies
273
274=========== 3.0.1
275        - PBSQuery
276                * Removed a testing code line
277                Reported By: sam @ vpac dot org
278                Fixed by: Bas van der Vlies
279
280=========== 3.0.0
281        - PBSQuery
282                * Cleaned up some old lines of code which didn't do anything.
283                * Changed the returnvalue of getqueue, getnode and getjob. Now
284                  they return an instance of the object, instead of a dict.
285                  This is API change so we bumped the version to 3.0.0
286                * Now the getnode-function accepts the short name as well as
287                  the long name as an argument.
288
289                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
290
291        - pbs.py, _pbs.so, PBSQuery.py
292                * Fixed a memory leak when used in daemon mode
293                Author: Bas van der Vlies <basv@sara.nl>
294
295=========== 2.9.8
296        - pbs_python.spec file patch added libdir and python defines so it
297          will build for RHEL5.1/CentOS 5.1
298          by        : Michael Sternberg <sternberg add anl dot gov>
299          applied by: Bas van der Vlies
300
301        - Updated the header files for pbs_python. So it can use some new
302          defines variables/defines, eg:
303                * ATTR_NODE_status
304                * ATTR_NODE_note
305                * ...
306          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
307
308        - Two new functions to log.h:
309                log_roll(int size);
310                log_size(void);
311          Added by: Bas van der Vlies
312
313        - We now have proper destructors (free memory) in SWIG for attrl and
314          attropl attributes.
315          Added by: Bas van der Vlies
316
317        - PBSQuery
318                * Added getnodes_with_property() function. This will get all nodes with
319                  a certain property, eg:
320                   - get all nodes with property 'cores8' and return all attributes:
321                      getnodes_with_property('cores8')
322               
323                   - get all nodes with property 'cores2' and only return the
324                     attributes 'state' and 'properties':
325                      attrl = [ 'state', 'properties' ]
326                      getnodes_with_property('cores2', attrl)
327               
328                     Only for torque version > 2.1.0
329             
330                Added by: Bas van der Vlies
331
332                * Added to class node the function
333                 - get_jobs(self, unique=None)
334                   Returns a list of the currently running job-id('s) on the node.
335                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
336
337                * Added get_server_name() function.
338                  This will return the PBS-server's name.
339                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
340
341                * Added to class _PBSobject the function
342                  - uniq(self, list)
343                    Filters out unique items of a list.
344                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
345
346                * Added to class job the function
347                  - get_nodes(self, unique=None)
348                    Returns a list of the nodes which run the job.
349                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
350
351=========== Version 2.9.4
352        -  Function pbs.pbs_geterrmsg does not exits any more is
353           replaced by pbs.error():
354                errno, text = pbs.error()
355                print errno, text
356           Reported by: pk at q-leap dot com
357           Fixed by   : Bas van der Vlies
358
359        - PBSQuery.py
360           * Added get_version() for server class, returns the version of
361             the batch server
362             Added by    : Bas van der Vlies
363
364           * Added get_nodes() for job class, returns a list of nodes on
365             which the job is run
366             Added by    : Bas van der Vlies
367
368           * PBSQuery.py rewrite make use of UserDict module.
369             Suggested by: Ramon Bastiaans
370             Fixed by    : Bas van der Vlies
371
372=========== Version 2.9.2
373        - The function pbs_rescquery() did not work. Fixed it
374        - The function avail(), did not work fixed it
375        - The functions pbs_rescreserve and pbs_rescrelease did not work
376          fixed it
377
378        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
379          nodes was wrong and state down has topmost priority
380
381=========== Version 2.9.0
382        - Added support for new torque build system 2.1.0
383        - Added new_rack_pbsmon.py
384          Authors: Walter de Jong & Bas van der Vlies
385
386=========== Version 2.8.2
387
388Configure has a new option:
389        --with-pbsdir=PATH
390
391If set it will use PATH specified as argument to
392find the pbs/torque libraries. If unset it will use
393some useful defaults to find the libaries.
394Suggested by: Troy Baer
395
396PBSQuery.py:
397        - If we can not make a connection with the server
398          then raise an exception and let the programmer
399          decides what to do.
400
401pbs_python.spec:
402        - It can now handle mode bits
403          Author: Martin Pels from SARA
404       
405=========== Version 2.8.0
406
407setup.py:
408        - Added /usr/lib to PBS_LIB_DIR
409
410pbs.py:
411        - Added new function "pbs.error()". This function checks if
412          an error has occured with a pbs function, eg:
413            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
414            error_number, err_txt = pbs.error()
415            if error_number:
416                print error_number, err_txt
417
418          prints the following message if script A.tsk does not exists:
419            15042 (qsub) cannot access script file
420
421pbs_python.spec:
422        - to make rpm packages from the source package, initial version
423        Author: Martin Pels from SARA
424
425README:
426        - Now requires python 2.1 and higher
427        - Explain how to build DEBIAN package
428
429=========== Version 2.7.10
430
431PBSQuery.py:
432        Fixed an error. You always got all resources from eg nodes
433        even when you supplied an attribute list where you only
434        requested the 'state' of the node.
435
436        Updated the inline python documentation
437
438=========== Version 2.7.9
439
440Fixed an error the previous version was in DEBUG mode. So
441you get an lot undesired output on the screen.
442
443debian package now also generated a package with the write
444version info.
445
446=========== Version 2.7.8
447
448Setup.py:
449        Can also handle ROCKS installations + the version
450        info is now the same as pbs.py
451        Thanks to: Roy Dragseth
452
453=========== Version 2.7.7
454
455PBSQuery module:
456  It can now be used in daemon programs. The pbs_server closes
457  the connection after an certain amount of time. This is fixed,
458  before every query there is an new connection and if the query
459  is finished the connection is closed.
460 
461=========== Version 2.7.6
462
463Now setup.py can also handle old Oscar Installations.
464Thanks To: Robin Humble
465
466Fixed an print statement fix in new_interface.py:
467Thanks To: Robin Humble
468
469Made some improvements for PBSQuery module:
470        - Updated the documentation
471        - Added the has_key() function to all
472          PBSObjects. So that the behaviour is similar to
473          an dictionary
474          Suggested By: Ramon Bastiaans
475       
476=========== Version 2.7.5
477 Added PBSQuery module.  This module requires pbs.py and it a simple module
478 for querying the pbs server. The documentation is in the module. Use for
479 example ipython to read it.
480
481 Written By: Roy Dragseth
482             Bas van der Vlies
483
484 eg: ( see also examples/new_interface.py)
485
486 from PBSQuery import PBSQuery
487 p = PBSQuery()
488
489 jobs = p.getjobs()
490 for name, job in jobs.items()
491        print job
492
493=========== Version 2.7.4
494 New versions fixes an bug in function pbs.pbs_statnode().
495 Reported by: Keith Poirier
496
497 Configure support:
498 Contributed By: Yaroslav Halchenko
499
500 Debian package support:
501 Contributed By: Yaroslav Halchenko
502
503 Added Support for Debuging the interface, You must edit pbs_wrap.c
504 and search for SARA_DEBUG:
505        #define SARA_DEBUG 1
506 Implemented by: Ramon Bastiaans
507
508 examples/pbsmon.py:
509        - Fixed an error when regex fails to determine node number,
510          no status was displayed
511        - It will now display and 'j' if the node is free for the batch
512          system and a job runs on the node (SMP-systems).
513 Changed by: Bas van der Vlies
514
515 examples/rack_pbsmon.py:
516        - An pbsmon that display node info per rack
517        - Edit the rack_pbsmon.py to adjust the values. It requires
518          that hostname contains rack and node id's,
519          eg: gb-r<number>n<number>
520 Contributed by: Walter de Jong
521
522=========== Version 2.7.3
523 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
524 Resource and QUEing manager).
525
526 The interface support OPENPBS and TORQUE.
527
528=========== Version 2.7.2
529 Some minor changes to include files. Now the Scalable PBS keywords
530 are also supported ( server attribute names: node_ping_rate and
531 node_check_rate). This does not interfere with the openpbs software.
532
533 pbsmon.py could not handle an one node cluster. Fixed it.
534
535=========== Version 2.7
536 Forgot to wrap the pbs_statfree() function. So we could not free
537 allocated memory from functions that return 'struct batch_status *'
538 like pbs_statjob().
539
540 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
541 Daniel Olson
542
543=========== Version 2.6.1
544 Setup.py now checks if all openpbs libraries are installed to compile the
545 package.
546
547=========== Version 2.6
548
549 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
550 function. This bug prevented to pass 'struct attrl' variables to this
551 function. Thanks to Evelyn Shiu for reporting this bug.
552
553
554=========== Version 2.5
555 Added the OpenPbs logging functions (log.h and liblog.a). There
556 is an example in the examples directory: logpbs.py
557
558 Fixed a bug in examples/resmom_info.py used the default port for
559 pbs_resmon if getservbyname fails.
560
561 Fixed a bug in resmom code. If the user has no permissions to query
562 the pbs_mom daemon a empty string is returned. Now we check for
563 the empty string. So we do not get a python exception.
564
565 Fixed a bug in setup.py forgot the exit statement when we did not
566 find the pbs libraries.
567
568=========== Version 2.3
569 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
570
571 Removed some obsolete code.
572
573 As for now we make use of the distutils module. So the package
574 is automatically compiled and installed in proper directories, usage:
575    python setup.py install
576 This requires that the package 'distutils' must be installed. As for
577 version 1.6 and higher it is in the distribution.
578
579 Added a function that prints the pbs python interface version. So we
580 can check if we have the right interface version:
581 import pbs
582 print pbs.version()
583 
584
585=========== Version 2.2
586
587In this release we can query the pbs_mom daemon with the aid of
588resource management functions:
589   pbs.openrm()
590   pbs.closerm()
591   pbs.downrm()
592   pbs.configrm()
593   pbs.addreq()
594   pbs.allreq()
595   pbs.flushreq()
596   pbs.activereq()
597   pbs.fullresp()
598   pbs.getreq()
599
600There is also a python function:
601   pbs.get_mom_values(id, [list]):
602     id   - connection id of pbs.openrm(node-name, port-number)
603     list - Is a optional paramter. A list of resource keywords
604
605   If 'list' is not supplied then this function will get the values
606   for the standard resource keywords, eg uname, loadave, .. + 'arch'
607   depended keywords. The arch feature is only implemented for linux,
608   but it can easily be ported to other oses.
609
610   if 'list' is supplied then we only fetch values for the keywords
611   that are in the list.
612
613   The function returns a dictonary. The keys are the resource keywords.
614
615   See 'resmom_info.py' for a example of this new feature.
616
617=========== Version 2.0
618
619The previous version was a simple interface above the PBS C API LIB.  This
620interface has gone a major changes. The most noticeable change is  that
621the functions accept/return Python lists instead of C-lists.  So you can now
622use standard Python syntax for manipulating Python lists. If you are familar
623with the Python syntax then you will appreciate this new interface.
624
625NOTE:
626  This interface is NOT compatible with the old one
627
628Here are some guidelines to convert your code to the new one:
629
630The constructors for the struct has gone a major change:
631  old code:
632    temp = pbs.new_attrl();
633    attrl_p = pbs.attrlPtr(temp)
634    attrl_p.name = 'state'
635
636  new_code:
637     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
638     attr_l[0].name = 'state'
639
640
641The pbs_stat functions returns Python lists instead of C-lists. There is
642NO next field anymore:
643  old_code:
644    temp = pbs.new_attrl();
645    attrl_p = pbs.attrlPtr(temp)
646    attrl_p.this = 'NULL'
647
648    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
649
650    while nodes.this != 'NULL':
651      print nodes.name
652
653      node_attrl = nodes.attribs
654      while node_attrl.this != 'NULL':
655        print '\t', node_attrl.name, '=', node_attrl.value
656        node_attrl = node_attrl.next
657
658    nodes = nodes.next
659
660
661  new_code:
662    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
663    for node in nodes:
664      print node.name
665      for attrib in node.attribs:
666        print '\t', attrib.name, '=', attrib.value
667
668Another advantage is you can use the print statement to show the
669connect of attrl and attropl classes:
670   attr_l = pbs.new_attrl(2)
671   attr_l[0].name = 'bas'
672   attr_l[0].value = 'van der Vlies'
673
674   print  attr_l[0]
675   >> (bas,,van der Vlies)
676   
677I hope these examples illustrate the changes. If you specify a wrong type
678for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.