source: trunk/CHANGES @ 270

Last change on this file since 270 was 270, checked in by bas, 13 years ago

fixed an error in examples/sara_nodes.py
  • Property svn:keywords set to Id
File size: 20.7 KB
Line 
1=========== 4.X.X
2 * New generated files for pbs_wrap.c and pbs.py to support python 3.X versions
3   Author: Bas van der Vlies
4
5 * Fixed AdvancedParser when using 01-12 range the zero was not appended.
6   Reported by: Ramon Bastiaans
7   Author: Dennis Stam
8
9 * examples/sara_nodes.py: Catch the PBSQuery error if we can not make a
10   connection with the batch server and exit the program.
11   Author: Bas van der Vlies
12
13=========== 4.3.0
14 * new_rack_pbsmon.py: For backward compatibility added -a/--all option.
15   Has the same functionality as -s/--summary
16   Author: Bas van der Vlies
17
18 * Added some license info to debian package and include GPL and LGPL 3.0
19   files.
20   Suggested by: W. Trevor King <wking at drexel dot edu>
21
22 * PBSQuery.py: Added support for new feature in node status field:
23    * message=EVENT:<key=value:>+
24    * message=ERROR <text>
25
26  Added the following syntax for node:
27        * node['error']
28    * node['event']
29
30  eg: node: n1
31    state = free
32    np = 8
33    properties = ib,switch1,highmem
34    ntype = cluster
35    jobs = 0/567403.sara.nl, 1/567403.sara.nl
36    status = ..,loadave=0.00,message=EVENT:sample.time=1288864220.003:cputotals.user=0:iconnect.pktout=0,netload=3487600394,state=free,...
37
38  example code:
39        p = PBSQuery()
40    node = p.getnode(n1)
41    print node.event['sample.time']
42    print node['event']['sample.time']
43
44   Requested by: Roy Dragseth <roy dot dragseth add uit dotno>
45   Author: Bas van der Vlies
46
47=========== 4.2.0
48 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
49   (jsbronder add gentoo dot org) for the information and
50   explanation.
51
52 * examples/pbsmon.py. Patched so it also works for torque 2.5
53   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
54   Applied by: Bas van der Vlies
55
56 * setup.py.in changed determine which torque version is installed
57   so we also support snapshots.
58   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
59   Applied by: Bas van der Vlies
60
61 * PBSQuery:
62   - get_nodes function for jobs now works for old and new datastructure and
63     gives correct results if unique=True
64     Reported by + idea for patch: Shunjie Lau (eijnuhs add gmail dot com)
65     Author: Bas van der Vlies
66
67 * Added AdvanceParser module. new_rack_pbsmon.py and sara_nodes.py make use of
68   this module
69   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
70
71 * examples/new_rack_pbsmon.py:
72     - Better specification for node expression
73     - added a new option -l/--list to list the state of a nodes and which job
74       is running on it, eg:
75             new_rack_pbsmon.py - l gb-r[1-3]n[3-6] gb-r4[n1-20]
76
77   Authors: Dennis Stam & Bas van der Vlies
78   Applied by: Bas van der Vlies
79
80 * examples/sara_nodes.py:
81   An utility to set/clear notes for multiple nodes, see ..
82   Author: Dennis Stam (Dennis dot Stam at sara dot nl)
83
84 * setup.py.in:
85   Added support for runtime-libraries when torque is installed in a non-default
86   location, closes #22
87   Author: Mattias (slabanja at chalmers dot se)
88   Applied by: Bas van der Vlies
89
90 * Changed build system for debian
91   Authors: Dennis Stam & Bas van der Vlies
92
93=========== 4.1.2
94 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
95   structure.
96   Author: Bas van der Vlies
97
98 * PBSQuery:
99     - fixed and error in get_jobs function for nodes. Did not
100       support new data structure and uniq did not work.
101       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
102
103=========== 4.1.0
104 * Delete function pbs.version() replaced by
105    * pbs.version
106        This is a string variable
107    * pbs.version_info
108        Is a tuple variable. So we can use it in scripts
109
110   This analogue with sys module, closes #20
111
112   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
113   Author: Bas van der Vlies
114
115 * More robust make/configure/setup.py
116   Reported by:
117        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
118        - Shunjie Lau <eijnuhs add gmail dot com>
119   Author: Bas van der Vlies
120
121
122 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
123   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
124   Author: Bas van der Vlies
125   
126=========== 4.0.2
127  * Quick fix so we do not get any undefined symbols. Forgot to
128    add the torque library when we compile the package.
129    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
130    Author: Bas van der Vlies
131
132=========== 4.0.1
133  * Fixed a bug in setting the right library path  for compiling.
134    Reported by: Chad Vizino ( vizino add psc dot edu )
135    Author: Bas van der Vlies
136   
137=========== 4.0.0
138  * pbs.py added some new functions for 2.4 version:
139    - pbs_fbserver(void);
140    - pbs_get_server_list(void);
141    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
142    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
143    - pbs_checkpointjob(int connect, char *job_id, char *extend);
144    - log_ext(int,char *,char *,int);
145    - log_init(char *, char *);
146    - log_remove_old(char *,unsigned long);
147
148    Note: version 2.1.X is still supported
149
150  * autotools environment
151    - Added support for finding pbs-config (borrowed for mpiexec source)
152    - Detecting which version of torque is installed
153    - only support installations that have pbs-config installed.
154    - Removed old code for installations that do not a have pbs-config
155      installed.
156
157  * PBSQuery.py
158    - There was a bug in the new data interface (Resource_List for jobs)
159      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
160      Applied by with some minor changes: Bas van der Vlies
161
162    - The new data structure is the default. You can switch back to the old
163      one with:
164            p = PBSQuery()
165            p.old_data_structure()
166
167      Author: Bas van der Vlies
168
169   * examples/ha_server.py
170     An example how to use the High Availabilty torque functions
171     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
172
173=========== 3.6.0
174  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
175    Reported by: Dennis Stam
176    Fixed by: Bas van der Vlies
177
178=========== 3.5.0
179  * PBSQuery
180        The class functions of node, job and queue support old and new data
181        structure.
182
183        Changed the behaviour of the new data stucture, We can use it as
184        dictionary and as class attribute, this is equivalent, eg:
185          - print node['np'] and print node.np
186
187        for a node we parse the 'status' line and split on '=' char, You now can
188        use these statements, eg
189          - print node.status.arch     (node['status'].arch or node['status']['arch'])
190          - print node.status.nsession
191
192        for a job we parse the 'Variable_List' line and split on '=' char, You now can
193        use the statements, eg:
194          - print job.Variable_List.PBS_O_WORKDIR
195          - print job.Variable_List.PBS_O_HOME
196
197        For more info see examples/new_interface.py
198
199    Author: Bas van der Vlies
200
201 * new_rack_pbsmon.py
202        Rewrite to new data structure and automatically determine how many nodes
203        and racks cluster has and skip printing of empty racks (default), use -w/--wide
204        for old behaviour.
205
206   Author: Bas van der Vlies
207
208
209=========== 3.2.0
210        - PBSQuery:
211          New data structure. In the old structure it is a dictionary
212          with a value and the value is a string. This is changed
213          that dictionary values are now of type list or dictionary depends
214          on the value of the keyword, eg for a node:
215            - np = 2:
216              * node['np'] = [ '2' ] 
217
218            - properties = cores2, mem4gb, parallel
219              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
220
221            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
222              * node['status']['arch'] = [ 'x86_64' ]
223              * node['status']['sessions'] = [ '222599' ]
224              * ...
225
226         The data structure is activated by the function:
227          * new_data_structure()
228
229        In a future release it will be come the default.
230        example:
231                p = PBSQuery()
232                p.new_data_structure()
233
234                nodes = p.getnodes()
235                print nodes.np, nodes['np']
236
237        Author: Bas van der Vlies
238
239        - PBSQuery
240          For old and new data structure we now can use another syntax:
241           * node['np'] and node.np are equivalent
242
243          This new syntax works for all keywords.
244
245        Author: Bas van der Vlies
246
247        - Added iter object for job, node, queue and server objects, eg:
248          node = p.getnode('gb-r1n1')
249          print node.name
250          for attrib in node:
251                print '%\t%s = %s' %(attrib, node[attrib])
252        Author: Bas van der Vlies
253
254        - fixed an error in getnode, getqueue and getjob, return empty
255          dictionary if not found
256        Author: Bas van der Vlies
257       
258        - New build system for rpm packages, make -f Makefile.rpm
259          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
260
261        Applied: Bas van der Vlies
262
263=========== 3.0.1
264        - PBSQuery
265                * Removed a testing code line
266                Reported By: sam @ vpac dot org
267                Fixed by: Bas van der Vlies
268
269=========== 3.0.0
270        - PBSQuery
271                * Cleaned up some old lines of code which didn't do anything.
272                * Changed the returnvalue of getqueue, getnode and getjob. Now
273                  they return an instance of the object, instead of a dict.
274                  This is API change so we bumped the version to 3.0.0
275                * Now the getnode-function accepts the short name as well as
276                  the long name as an argument.
277
278                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
279
280        - pbs.py, _pbs.so, PBSQuery.py
281                * Fixed a memory leak when used in daemon mode
282                Author: Bas van der Vlies <basv@sara.nl>
283
284=========== 2.9.8
285        - pbs_python.spec file patch added libdir and python defines so it
286          will build for RHEL5.1/CentOS 5.1
287          by        : Michael Sternberg <sternberg add anl dot gov>
288          applied by: Bas van der Vlies
289
290        - Updated the header files for pbs_python. So it can use some new
291          defines variables/defines, eg:
292                * ATTR_NODE_status
293                * ATTR_NODE_note
294                * ...
295          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
296
297        - Two new functions to log.h:
298                log_roll(int size);
299                log_size(void);
300          Added by: Bas van der Vlies
301
302        - We now have proper destructors (free memory) in SWIG for attrl and
303          attropl attributes.
304          Added by: Bas van der Vlies
305
306        - PBSQuery
307                * Added getnodes_with_property() function. This will get all nodes with
308                  a certain property, eg:
309                   - get all nodes with property 'cores8' and return all attributes:
310                      getnodes_with_property('cores8')
311               
312                   - get all nodes with property 'cores2' and only return the
313                     attributes 'state' and 'properties':
314                      attrl = [ 'state', 'properties' ]
315                      getnodes_with_property('cores2', attrl)
316               
317                     Only for torque version > 2.1.0
318             
319                Added by: Bas van der Vlies
320
321                * Added to class node the function
322                 - get_jobs(self, unique=None)
323                   Returns a list of the currently running job-id('s) on the node.
324                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
325
326                * Added get_server_name() function.
327                  This will return the PBS-server's name.
328                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
329
330                * Added to class _PBSobject the function
331                  - uniq(self, list)
332                    Filters out unique items of a list.
333                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
334
335                * Added to class job the function
336                  - get_nodes(self, unique=None)
337                    Returns a list of the nodes which run the job.
338                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
339
340=========== Version 2.9.4
341        -  Function pbs.pbs_geterrmsg does not exits any more is
342           replaced by pbs.error():
343                errno, text = pbs.error()
344                print errno, text
345           Reported by: pk at q-leap dot com
346           Fixed by   : Bas van der Vlies
347
348        - PBSQuery.py
349           * Added get_version() for server class, returns the version of
350             the batch server
351             Added by    : Bas van der Vlies
352
353           * Added get_nodes() for job class, returns a list of nodes on
354             which the job is run
355             Added by    : Bas van der Vlies
356
357           * PBSQuery.py rewrite make use of UserDict module.
358             Suggested by: Ramon Bastiaans
359             Fixed by    : Bas van der Vlies
360
361=========== Version 2.9.2
362        - The function pbs_rescquery() did not work. Fixed it
363        - The function avail(), did not work fixed it
364        - The functions pbs_rescreserve and pbs_rescrelease did not work
365          fixed it
366
367        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
368          nodes was wrong and state down has topmost priority
369
370=========== Version 2.9.0
371        - Added support for new torque build system 2.1.0
372        - Added new_rack_pbsmon.py
373          Authors: Walter de Jong & Bas van der Vlies
374
375=========== Version 2.8.2
376
377Configure has a new option:
378        --with-pbsdir=PATH
379
380If set it will use PATH specified as argument to
381find the pbs/torque libraries. If unset it will use
382some useful defaults to find the libaries.
383Suggested by: Troy Baer
384
385PBSQuery.py:
386        - If we can not make a connection with the server
387          then raise an exception and let the programmer
388          decides what to do.
389
390pbs_python.spec:
391        - It can now handle mode bits
392          Author: Martin Pels from SARA
393       
394=========== Version 2.8.0
395
396setup.py:
397        - Added /usr/lib to PBS_LIB_DIR
398
399pbs.py:
400        - Added new function "pbs.error()". This function checks if
401          an error has occured with a pbs function, eg:
402            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
403            error_number, err_txt = pbs.error()
404            if error_number:
405                print error_number, err_txt
406
407          prints the following message if script A.tsk does not exists:
408            15042 (qsub) cannot access script file
409
410pbs_python.spec:
411        - to make rpm packages from the source package, initial version
412        Author: Martin Pels from SARA
413
414README:
415        - Now requires python 2.1 and higher
416        - Explain how to build DEBIAN package
417
418=========== Version 2.7.10
419
420PBSQuery.py:
421        Fixed an error. You always got all resources from eg nodes
422        even when you supplied an attribute list where you only
423        requested the 'state' of the node.
424
425        Updated the inline python documentation
426
427=========== Version 2.7.9
428
429Fixed an error the previous version was in DEBUG mode. So
430you get an lot undesired output on the screen.
431
432debian package now also generated a package with the write
433version info.
434
435=========== Version 2.7.8
436
437Setup.py:
438        Can also handle ROCKS installations + the version
439        info is now the same as pbs.py
440        Thanks to: Roy Dragseth
441
442=========== Version 2.7.7
443
444PBSQuery module:
445  It can now be used in daemon programs. The pbs_server closes
446  the connection after an certain amount of time. This is fixed,
447  before every query there is an new connection and if the query
448  is finished the connection is closed.
449 
450=========== Version 2.7.6
451
452Now setup.py can also handle old Oscar Installations.
453Thanks To: Robin Humble
454
455Fixed an print statement fix in new_interface.py:
456Thanks To: Robin Humble
457
458Made some improvements for PBSQuery module:
459        - Updated the documentation
460        - Added the has_key() function to all
461          PBSObjects. So that the behaviour is similar to
462          an dictionary
463          Suggested By: Ramon Bastiaans
464       
465=========== Version 2.7.5
466 Added PBSQuery module.  This module requires pbs.py and it a simple module
467 for querying the pbs server. The documentation is in the module. Use for
468 example ipython to read it.
469
470 Written By: Roy Dragseth
471             Bas van der Vlies
472
473 eg: ( see also examples/new_interface.py)
474
475 from PBSQuery import PBSQuery
476 p = PBSQuery()
477
478 jobs = p.getjobs()
479 for name, job in jobs.items()
480        print job
481
482=========== Version 2.7.4
483 New versions fixes an bug in function pbs.pbs_statnode().
484 Reported by: Keith Poirier
485
486 Configure support:
487 Contributed By: Yaroslav Halchenko
488
489 Debian package support:
490 Contributed By: Yaroslav Halchenko
491
492 Added Support for Debuging the interface, You must edit pbs_wrap.c
493 and search for SARA_DEBUG:
494        #define SARA_DEBUG 1
495 Implemented by: Ramon Bastiaans
496
497 examples/pbsmon.py:
498        - Fixed an error when regex fails to determine node number,
499          no status was displayed
500        - It will now display and 'j' if the node is free for the batch
501          system and a job runs on the node (SMP-systems).
502 Changed by: Bas van der Vlies
503
504 examples/rack_pbsmon.py:
505        - An pbsmon that display node info per rack
506        - Edit the rack_pbsmon.py to adjust the values. It requires
507          that hostname contains rack and node id's,
508          eg: gb-r<number>n<number>
509 Contributed by: Walter de Jong
510
511=========== Version 2.7.3
512 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
513 Resource and QUEing manager).
514
515 The interface support OPENPBS and TORQUE.
516
517=========== Version 2.7.2
518 Some minor changes to include files. Now the Scalable PBS keywords
519 are also supported ( server attribute names: node_ping_rate and
520 node_check_rate). This does not interfere with the openpbs software.
521
522 pbsmon.py could not handle an one node cluster. Fixed it.
523
524=========== Version 2.7
525 Forgot to wrap the pbs_statfree() function. So we could not free
526 allocated memory from functions that return 'struct batch_status *'
527 like pbs_statjob().
528
529 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
530 Daniel Olson
531
532=========== Version 2.6.1
533 Setup.py now checks if all openpbs libraries are installed to compile the
534 package.
535
536=========== Version 2.6
537
538 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
539 function. This bug prevented to pass 'struct attrl' variables to this
540 function. Thanks to Evelyn Shiu for reporting this bug.
541
542
543=========== Version 2.5
544 Added the OpenPbs logging functions (log.h and liblog.a). There
545 is an example in the examples directory: logpbs.py
546
547 Fixed a bug in examples/resmom_info.py used the default port for
548 pbs_resmon if getservbyname fails.
549
550 Fixed a bug in resmom code. If the user has no permissions to query
551 the pbs_mom daemon a empty string is returned. Now we check for
552 the empty string. So we do not get a python exception.
553
554 Fixed a bug in setup.py forgot the exit statement when we did not
555 find the pbs libraries.
556
557=========== Version 2.3
558 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
559
560 Removed some obsolete code.
561
562 As for now we make use of the distutils module. So the package
563 is automatically compiled and installed in proper directories, usage:
564    python setup.py install
565 This requires that the package 'distutils' must be installed. As for
566 version 1.6 and higher it is in the distribution.
567
568 Added a function that prints the pbs python interface version. So we
569 can check if we have the right interface version:
570 import pbs
571 print pbs.version()
572 
573
574=========== Version 2.2
575
576In this release we can query the pbs_mom daemon with the aid of
577resource management functions:
578   pbs.openrm()
579   pbs.closerm()
580   pbs.downrm()
581   pbs.configrm()
582   pbs.addreq()
583   pbs.allreq()
584   pbs.flushreq()
585   pbs.activereq()
586   pbs.fullresp()
587   pbs.getreq()
588
589There is also a python function:
590   pbs.get_mom_values(id, [list]):
591     id   - connection id of pbs.openrm(node-name, port-number)
592     list - Is a optional paramter. A list of resource keywords
593
594   If 'list' is not supplied then this function will get the values
595   for the standard resource keywords, eg uname, loadave, .. + 'arch'
596   depended keywords. The arch feature is only implemented for linux,
597   but it can easily be ported to other oses.
598
599   if 'list' is supplied then we only fetch values for the keywords
600   that are in the list.
601
602   The function returns a dictonary. The keys are the resource keywords.
603
604   See 'resmom_info.py' for a example of this new feature.
605
606=========== Version 2.0
607
608The previous version was a simple interface above the PBS C API LIB.  This
609interface has gone a major changes. The most noticeable change is  that
610the functions accept/return Python lists instead of C-lists.  So you can now
611use standard Python syntax for manipulating Python lists. If you are familar
612with the Python syntax then you will appreciate this new interface.
613
614NOTE:
615  This interface is NOT compatible with the old one
616
617Here are some guidelines to convert your code to the new one:
618
619The constructors for the struct has gone a major change:
620  old code:
621    temp = pbs.new_attrl();
622    attrl_p = pbs.attrlPtr(temp)
623    attrl_p.name = 'state'
624
625  new_code:
626     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
627     attr_l[0].name = 'state'
628
629
630The pbs_stat functions returns Python lists instead of C-lists. There is
631NO next field anymore:
632  old_code:
633    temp = pbs.new_attrl();
634    attrl_p = pbs.attrlPtr(temp)
635    attrl_p.this = 'NULL'
636
637    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
638
639    while nodes.this != 'NULL':
640      print nodes.name
641
642      node_attrl = nodes.attribs
643      while node_attrl.this != 'NULL':
644        print '\t', node_attrl.name, '=', node_attrl.value
645        node_attrl = node_attrl.next
646
647    nodes = nodes.next
648
649
650  new_code:
651    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
652    for node in nodes:
653      print node.name
654      for attrib in node.attribs:
655        print '\t', attrib.name, '=', attrib.value
656
657Another advantage is you can use the print statement to show the
658connect of attrl and attropl classes:
659   attr_l = pbs.new_attrl(2)
660   attr_l[0].name = 'bas'
661   attr_l[0].value = 'van der Vlies'
662
663   print  attr_l[0]
664   >> (bas,,van der Vlies)
665   
666I hope these examples illustrate the changes. If you specify a wrong type
667for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.