source: trunk/CHANGES @ 284

Last change on this file since 284 was 282, checked in by bas, 12 years ago

renamed some file and added documentation
  • Property svn:keywords set to Id
File size: 21.6 KB
Line 
1=========== 4.3.4
2 * PBSQuery bugfix. getjob() can now handle short jobname, eg:
3    * p.getjob('6073545')
4
5   Author: Willem Vermin (SARA) & Bas van der Vlies
6
7 * added an example: pbs_jobmonitor shows the output of top
8   -u user on the node
9    - one cycle of top
10    - user: the user the job belongs to
11
12   renamed as pbs_joblogin logs in to the node as the user
13   who invokes this script (os.getenv('USER'))
14
15   Author: Willem Vermin (SARA)
16 
17=========== 4.3.3
18 * New generated files for pbs_wrap.c and pbs.py to support python 3.X versions
19   Reported by: Steve Traylen <steve dot traylen add cern dot ch>
20   Author: Bas van der Vlies
21
22 * Fixed AdvancedParser when using 01-12 range the zero was not appended.
23   Reported by: Ramon Bastiaans
24   Author: Dennis Stam
25
26 * examples/sara_nodes.py: Catch the PBSQuery error if we can not make a
27   connection with the batch server and exit the program.
28   Author: Bas van der Vlies
29
30 * Remove the debian dependency on AdvancedParser. We now have
31   our own PBSAdvanceParser so it does not conflict with other SARA
32   packages.
33   Reported by: Ramon Bastiaans (SARA)
34   Fixed by: Bas van der Vlies
35
36 * Make pbs_torque only dependend on libtorque instead of torque and adjusted
37   the maintainer of the package, closes #30
38   Reported by: Guillermo Marcus  <guillermo dot marcus add@gmail dot com>
39   Fixed by: Bas van der Vlies
40   
41=========== 4.3.0
42 * new_rack_pbsmon.py: For backward compatibility added -a/--all option.
43   Has the same functionality as -s/--summary
44   Author: Bas van der Vlies
45
46 * Added some license info to debian package and include GPL and LGPL 3.0
47   files.
48   Suggested by: W. Trevor King <wking at drexel dot edu>
49
50 * PBSQuery.py: Added support for new feature in node status field:
51    * message=EVENT:<key=value:>+
52    * message=ERROR <text>
53
54  Added the following syntax for node:
55        * node['error']
56    * node['event']
57
58  eg: node: n1
59    state = free
60    np = 8
61    properties = ib,switch1,highmem
62    ntype = cluster
63    jobs = 0/567403.sara.nl, 1/567403.sara.nl
64    status = ..,loadave=0.00,message=EVENT:sample.time=1288864220.003:cputotals.user=0:iconnect.pktout=0,netload=3487600394,state=free,...
65
66  example code:
67        p = PBSQuery()
68    node = p.getnode(n1)
69    print node.event['sample.time']
70    print node['event']['sample.time']
71
72   Requested by: Roy Dragseth <roy dot dragseth add uit dotno>
73   Author: Bas van der Vlies
74
75=========== 4.2.0
76 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
77   (jsbronder add gentoo dot org) for the information and
78   explanation.
79
80 * examples/pbsmon.py. Patched so it also works for torque 2.5
81   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
82   Applied by: Bas van der Vlies
83
84 * setup.py.in changed determine which torque version is installed
85   so we also support snapshots.
86   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
87   Applied by: Bas van der Vlies
88
89 * PBSQuery:
90   - get_nodes function for jobs now works for old and new datastructure and
91     gives correct results if unique=True
92     Reported by + idea for patch: Shunjie Lau (eijnuhs add gmail dot com)
93     Author: Bas van der Vlies
94
95 * Added AdvanceParser module. new_rack_pbsmon.py and sara_nodes.py make use of
96   this module
97   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
98
99 * examples/new_rack_pbsmon.py:
100     - Better specification for node expression
101     - added a new option -l/--list to list the state of a nodes and which job
102       is running on it, eg:
103             new_rack_pbsmon.py - l gb-r[1-3]n[3-6] gb-r4[n1-20]
104
105   Authors: Dennis Stam & Bas van der Vlies
106   Applied by: Bas van der Vlies
107
108 * examples/sara_nodes.py:
109   An utility to set/clear notes for multiple nodes, see ..
110   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
111
112 * setup.py.in:
113   Added support for runtime-libraries when torque is installed in a non-default
114   location, closes #22
115   Author: Mattias (slabanja at chalmers dot se)
116   Applied by: Bas van der Vlies
117
118 * Changed build system for debian
119   Authors: Dennis Stam & Bas van der Vlies
120
121=========== 4.1.2
122 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
123   structure.
124   Author: Bas van der Vlies
125
126 * PBSQuery:
127     - fixed and error in get_jobs function for nodes. Did not
128       support new data structure and uniq did not work.
129       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
130
131=========== 4.1.0
132 * Delete function pbs.version() replaced by
133    * pbs.version
134        This is a string variable
135    * pbs.version_info
136        Is a tuple variable. So we can use it in scripts
137
138   This analogue with sys module, closes #20
139
140   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
141   Author: Bas van der Vlies
142
143 * More robust make/configure/setup.py
144   Reported by:
145        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
146        - Shunjie Lau <eijnuhs add gmail dot com>
147   Author: Bas van der Vlies
148
149
150 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
151   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
152   Author: Bas van der Vlies
153   
154=========== 4.0.2
155  * Quick fix so we do not get any undefined symbols. Forgot to
156    add the torque library when we compile the package.
157    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
158    Author: Bas van der Vlies
159
160=========== 4.0.1
161  * Fixed a bug in setting the right library path  for compiling.
162    Reported by: Chad Vizino ( vizino add psc dot edu )
163    Author: Bas van der Vlies
164   
165=========== 4.0.0
166  * pbs.py added some new functions for 2.4 version:
167    - pbs_fbserver(void);
168    - pbs_get_server_list(void);
169    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
170    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
171    - pbs_checkpointjob(int connect, char *job_id, char *extend);
172    - log_ext(int,char *,char *,int);
173    - log_init(char *, char *);
174    - log_remove_old(char *,unsigned long);
175
176    Note: version 2.1.X is still supported
177
178  * autotools environment
179    - Added support for finding pbs-config (borrowed for mpiexec source)
180    - Detecting which version of torque is installed
181    - only support installations that have pbs-config installed.
182    - Removed old code for installations that do not a have pbs-config
183      installed.
184
185  * PBSQuery.py
186    - There was a bug in the new data interface (Resource_List for jobs)
187      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
188      Applied by with some minor changes: Bas van der Vlies
189
190    - The new data structure is the default. You can switch back to the old
191      one with:
192            p = PBSQuery()
193            p.old_data_structure()
194
195      Author: Bas van der Vlies
196
197   * examples/ha_server.py
198     An example how to use the High Availabilty torque functions
199     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
200
201=========== 3.6.0
202  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
203    Reported by: Dennis Stam
204    Fixed by: Bas van der Vlies
205
206=========== 3.5.0
207  * PBSQuery
208        The class functions of node, job and queue support old and new data
209        structure.
210
211        Changed the behaviour of the new data stucture, We can use it as
212        dictionary and as class attribute, this is equivalent, eg:
213          - print node['np'] and print node.np
214
215        for a node we parse the 'status' line and split on '=' char, You now can
216        use these statements, eg
217          - print node.status.arch     (node['status'].arch or node['status']['arch'])
218          - print node.status.nsession
219
220        for a job we parse the 'Variable_List' line and split on '=' char, You now can
221        use the statements, eg:
222          - print job.Variable_List.PBS_O_WORKDIR
223          - print job.Variable_List.PBS_O_HOME
224
225        For more info see examples/new_interface.py
226
227    Author: Bas van der Vlies
228
229 * new_rack_pbsmon.py
230        Rewrite to new data structure and automatically determine how many nodes
231        and racks cluster has and skip printing of empty racks (default), use -w/--wide
232        for old behaviour.
233
234   Author: Bas van der Vlies
235
236
237=========== 3.2.0
238        - PBSQuery:
239          New data structure. In the old structure it is a dictionary
240          with a value and the value is a string. This is changed
241          that dictionary values are now of type list or dictionary depends
242          on the value of the keyword, eg for a node:
243            - np = 2:
244              * node['np'] = [ '2' ] 
245
246            - properties = cores2, mem4gb, parallel
247              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
248
249            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
250              * node['status']['arch'] = [ 'x86_64' ]
251              * node['status']['sessions'] = [ '222599' ]
252              * ...
253
254         The data structure is activated by the function:
255          * new_data_structure()
256
257        In a future release it will be come the default.
258        example:
259                p = PBSQuery()
260                p.new_data_structure()
261
262                nodes = p.getnodes()
263                print nodes.np, nodes['np']
264
265        Author: Bas van der Vlies
266
267        - PBSQuery
268          For old and new data structure we now can use another syntax:
269           * node['np'] and node.np are equivalent
270
271          This new syntax works for all keywords.
272
273        Author: Bas van der Vlies
274
275        - Added iter object for job, node, queue and server objects, eg:
276          node = p.getnode('gb-r1n1')
277          print node.name
278          for attrib in node:
279                print '%\t%s = %s' %(attrib, node[attrib])
280        Author: Bas van der Vlies
281
282        - fixed an error in getnode, getqueue and getjob, return empty
283          dictionary if not found
284        Author: Bas van der Vlies
285       
286        - New build system for rpm packages, make -f Makefile.rpm
287          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
288
289        Applied: Bas van der Vlies
290
291=========== 3.0.1
292        - PBSQuery
293                * Removed a testing code line
294                Reported By: sam @ vpac dot org
295                Fixed by: Bas van der Vlies
296
297=========== 3.0.0
298        - PBSQuery
299                * Cleaned up some old lines of code which didn't do anything.
300                * Changed the returnvalue of getqueue, getnode and getjob. Now
301                  they return an instance of the object, instead of a dict.
302                  This is API change so we bumped the version to 3.0.0
303                * Now the getnode-function accepts the short name as well as
304                  the long name as an argument.
305
306                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
307
308        - pbs.py, _pbs.so, PBSQuery.py
309                * Fixed a memory leak when used in daemon mode
310                Author: Bas van der Vlies <basv@sara.nl>
311
312=========== 2.9.8
313        - pbs_python.spec file patch added libdir and python defines so it
314          will build for RHEL5.1/CentOS 5.1
315          by        : Michael Sternberg <sternberg add anl dot gov>
316          applied by: Bas van der Vlies
317
318        - Updated the header files for pbs_python. So it can use some new
319          defines variables/defines, eg:
320                * ATTR_NODE_status
321                * ATTR_NODE_note
322                * ...
323          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
324
325        - Two new functions to log.h:
326                log_roll(int size);
327                log_size(void);
328          Added by: Bas van der Vlies
329
330        - We now have proper destructors (free memory) in SWIG for attrl and
331          attropl attributes.
332          Added by: Bas van der Vlies
333
334        - PBSQuery
335                * Added getnodes_with_property() function. This will get all nodes with
336                  a certain property, eg:
337                   - get all nodes with property 'cores8' and return all attributes:
338                      getnodes_with_property('cores8')
339               
340                   - get all nodes with property 'cores2' and only return the
341                     attributes 'state' and 'properties':
342                      attrl = [ 'state', 'properties' ]
343                      getnodes_with_property('cores2', attrl)
344               
345                     Only for torque version > 2.1.0
346             
347                Added by: Bas van der Vlies
348
349                * Added to class node the function
350                 - get_jobs(self, unique=None)
351                   Returns a list of the currently running job-id('s) on the node.
352                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
353
354                * Added get_server_name() function.
355                  This will return the PBS-server's name.
356                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
357
358                * Added to class _PBSobject the function
359                  - uniq(self, list)
360                    Filters out unique items of a list.
361                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
362
363                * Added to class job the function
364                  - get_nodes(self, unique=None)
365                    Returns a list of the nodes which run the job.
366                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
367
368=========== Version 2.9.4
369        -  Function pbs.pbs_geterrmsg does not exits any more is
370           replaced by pbs.error():
371                errno, text = pbs.error()
372                print errno, text
373           Reported by: pk at q-leap dot com
374           Fixed by   : Bas van der Vlies
375
376        - PBSQuery.py
377           * Added get_version() for server class, returns the version of
378             the batch server
379             Added by    : Bas van der Vlies
380
381           * Added get_nodes() for job class, returns a list of nodes on
382             which the job is run
383             Added by    : Bas van der Vlies
384
385           * PBSQuery.py rewrite make use of UserDict module.
386             Suggested by: Ramon Bastiaans
387             Fixed by    : Bas van der Vlies
388
389=========== Version 2.9.2
390        - The function pbs_rescquery() did not work. Fixed it
391        - The function avail(), did not work fixed it
392        - The functions pbs_rescreserve and pbs_rescrelease did not work
393          fixed it
394
395        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
396          nodes was wrong and state down has topmost priority
397
398=========== Version 2.9.0
399        - Added support for new torque build system 2.1.0
400        - Added new_rack_pbsmon.py
401          Authors: Walter de Jong & Bas van der Vlies
402
403=========== Version 2.8.2
404
405Configure has a new option:
406        --with-pbsdir=PATH
407
408If set it will use PATH specified as argument to
409find the pbs/torque libraries. If unset it will use
410some useful defaults to find the libaries.
411Suggested by: Troy Baer
412
413PBSQuery.py:
414        - If we can not make a connection with the server
415          then raise an exception and let the programmer
416          decides what to do.
417
418pbs_python.spec:
419        - It can now handle mode bits
420          Author: Martin Pels from SARA
421       
422=========== Version 2.8.0
423
424setup.py:
425        - Added /usr/lib to PBS_LIB_DIR
426
427pbs.py:
428        - Added new function "pbs.error()". This function checks if
429          an error has occured with a pbs function, eg:
430            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
431            error_number, err_txt = pbs.error()
432            if error_number:
433                print error_number, err_txt
434
435          prints the following message if script A.tsk does not exists:
436            15042 (qsub) cannot access script file
437
438pbs_python.spec:
439        - to make rpm packages from the source package, initial version
440        Author: Martin Pels from SARA
441
442README:
443        - Now requires python 2.1 and higher
444        - Explain how to build DEBIAN package
445
446=========== Version 2.7.10
447
448PBSQuery.py:
449        Fixed an error. You always got all resources from eg nodes
450        even when you supplied an attribute list where you only
451        requested the 'state' of the node.
452
453        Updated the inline python documentation
454
455=========== Version 2.7.9
456
457Fixed an error the previous version was in DEBUG mode. So
458you get an lot undesired output on the screen.
459
460debian package now also generated a package with the write
461version info.
462
463=========== Version 2.7.8
464
465Setup.py:
466        Can also handle ROCKS installations + the version
467        info is now the same as pbs.py
468        Thanks to: Roy Dragseth
469
470=========== Version 2.7.7
471
472PBSQuery module:
473  It can now be used in daemon programs. The pbs_server closes
474  the connection after an certain amount of time. This is fixed,
475  before every query there is an new connection and if the query
476  is finished the connection is closed.
477 
478=========== Version 2.7.6
479
480Now setup.py can also handle old Oscar Installations.
481Thanks To: Robin Humble
482
483Fixed an print statement fix in new_interface.py:
484Thanks To: Robin Humble
485
486Made some improvements for PBSQuery module:
487        - Updated the documentation
488        - Added the has_key() function to all
489          PBSObjects. So that the behaviour is similar to
490          an dictionary
491          Suggested By: Ramon Bastiaans
492       
493=========== Version 2.7.5
494 Added PBSQuery module.  This module requires pbs.py and it a simple module
495 for querying the pbs server. The documentation is in the module. Use for
496 example ipython to read it.
497
498 Written By: Roy Dragseth
499             Bas van der Vlies
500
501 eg: ( see also examples/new_interface.py)
502
503 from PBSQuery import PBSQuery
504 p = PBSQuery()
505
506 jobs = p.getjobs()
507 for name, job in jobs.items()
508        print job
509
510=========== Version 2.7.4
511 New versions fixes an bug in function pbs.pbs_statnode().
512 Reported by: Keith Poirier
513
514 Configure support:
515 Contributed By: Yaroslav Halchenko
516
517 Debian package support:
518 Contributed By: Yaroslav Halchenko
519
520 Added Support for Debuging the interface, You must edit pbs_wrap.c
521 and search for SARA_DEBUG:
522        #define SARA_DEBUG 1
523 Implemented by: Ramon Bastiaans
524
525 examples/pbsmon.py:
526        - Fixed an error when regex fails to determine node number,
527          no status was displayed
528        - It will now display and 'j' if the node is free for the batch
529          system and a job runs on the node (SMP-systems).
530 Changed by: Bas van der Vlies
531
532 examples/rack_pbsmon.py:
533        - An pbsmon that display node info per rack
534        - Edit the rack_pbsmon.py to adjust the values. It requires
535          that hostname contains rack and node id's,
536          eg: gb-r<number>n<number>
537 Contributed by: Walter de Jong
538
539=========== Version 2.7.3
540 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
541 Resource and QUEing manager).
542
543 The interface support OPENPBS and TORQUE.
544
545=========== Version 2.7.2
546 Some minor changes to include files. Now the Scalable PBS keywords
547 are also supported ( server attribute names: node_ping_rate and
548 node_check_rate). This does not interfere with the openpbs software.
549
550 pbsmon.py could not handle an one node cluster. Fixed it.
551
552=========== Version 2.7
553 Forgot to wrap the pbs_statfree() function. So we could not free
554 allocated memory from functions that return 'struct batch_status *'
555 like pbs_statjob().
556
557 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
558 Daniel Olson
559
560=========== Version 2.6.1
561 Setup.py now checks if all openpbs libraries are installed to compile the
562 package.
563
564=========== Version 2.6
565
566 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
567 function. This bug prevented to pass 'struct attrl' variables to this
568 function. Thanks to Evelyn Shiu for reporting this bug.
569
570
571=========== Version 2.5
572 Added the OpenPbs logging functions (log.h and liblog.a). There
573 is an example in the examples directory: logpbs.py
574
575 Fixed a bug in examples/resmom_info.py used the default port for
576 pbs_resmon if getservbyname fails.
577
578 Fixed a bug in resmom code. If the user has no permissions to query
579 the pbs_mom daemon a empty string is returned. Now we check for
580 the empty string. So we do not get a python exception.
581
582 Fixed a bug in setup.py forgot the exit statement when we did not
583 find the pbs libraries.
584
585=========== Version 2.3
586 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
587
588 Removed some obsolete code.
589
590 As for now we make use of the distutils module. So the package
591 is automatically compiled and installed in proper directories, usage:
592    python setup.py install
593 This requires that the package 'distutils' must be installed. As for
594 version 1.6 and higher it is in the distribution.
595
596 Added a function that prints the pbs python interface version. So we
597 can check if we have the right interface version:
598 import pbs
599 print pbs.version()
600 
601
602=========== Version 2.2
603
604In this release we can query the pbs_mom daemon with the aid of
605resource management functions:
606   pbs.openrm()
607   pbs.closerm()
608   pbs.downrm()
609   pbs.configrm()
610   pbs.addreq()
611   pbs.allreq()
612   pbs.flushreq()
613   pbs.activereq()
614   pbs.fullresp()
615   pbs.getreq()
616
617There is also a python function:
618   pbs.get_mom_values(id, [list]):
619     id   - connection id of pbs.openrm(node-name, port-number)
620     list - Is a optional paramter. A list of resource keywords
621
622   If 'list' is not supplied then this function will get the values
623   for the standard resource keywords, eg uname, loadave, .. + 'arch'
624   depended keywords. The arch feature is only implemented for linux,
625   but it can easily be ported to other oses.
626
627   if 'list' is supplied then we only fetch values for the keywords
628   that are in the list.
629
630   The function returns a dictonary. The keys are the resource keywords.
631
632   See 'resmom_info.py' for a example of this new feature.
633
634=========== Version 2.0
635
636The previous version was a simple interface above the PBS C API LIB.  This
637interface has gone a major changes. The most noticeable change is  that
638the functions accept/return Python lists instead of C-lists.  So you can now
639use standard Python syntax for manipulating Python lists. If you are familar
640with the Python syntax then you will appreciate this new interface.
641
642NOTE:
643  This interface is NOT compatible with the old one
644
645Here are some guidelines to convert your code to the new one:
646
647The constructors for the struct has gone a major change:
648  old code:
649    temp = pbs.new_attrl();
650    attrl_p = pbs.attrlPtr(temp)
651    attrl_p.name = 'state'
652
653  new_code:
654     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
655     attr_l[0].name = 'state'
656
657
658The pbs_stat functions returns Python lists instead of C-lists. There is
659NO next field anymore:
660  old_code:
661    temp = pbs.new_attrl();
662    attrl_p = pbs.attrlPtr(temp)
663    attrl_p.this = 'NULL'
664
665    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
666
667    while nodes.this != 'NULL':
668      print nodes.name
669
670      node_attrl = nodes.attribs
671      while node_attrl.this != 'NULL':
672        print '\t', node_attrl.name, '=', node_attrl.value
673        node_attrl = node_attrl.next
674
675    nodes = nodes.next
676
677
678  new_code:
679    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
680    for node in nodes:
681      print node.name
682      for attrib in node.attribs:
683        print '\t', attrib.name, '=', attrib.value
684
685Another advantage is you can use the print statement to show the
686connect of attrl and attropl classes:
687   attr_l = pbs.new_attrl(2)
688   attr_l[0].name = 'bas'
689   attr_l[0].value = 'van der Vlies'
690
691   print  attr_l[0]
692   >> (bas,,van der Vlies)
693   
694I hope these examples illustrate the changes. If you specify a wrong type
695for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.