source: trunk/CHANGES @ 246

Last change on this file since 246 was 246, checked in by bas, 14 years ago

get_nodes for job now works for new and old structure and fixed the unique output
  • Property svn:keywords set to Id
File size: 18.3 KB
Line 
1=========== 4.1.3
2 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
3  (jsbronder add gentoo dot org) for the information and
4  explanation.
5
6 * examples/pbsmon.py. Patched so it also works for torque 2.5
7   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
8   Applied by: Bas van der Vlies
9
10 * setup.py.in changed determine which torque version is installed
11   so we also support snapshots.
12   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
13   Applied by: Bas van der Vlies
14
15 * get_nodes function for jobs now works for old and new datastructure and
16   gives correct results if unique=True
17   Author: Bas van der Vlies
18
19=========== 4.1.2
20 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
21   structure.
22   Author: Bas van der Vlies
23
24 * PBSQuery:
25     - fixed and error in get_jobs function for nodes. Did not
26       support new data structure and uniq did not work.
27       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
28
29=========== 4.1.0
30 * Delete function pbs.version() replaced by
31    * pbs.version
32        This is a string variable
33    * pbs.version_info
34        Is a tuple variable. So we can use it in scripts
35
36   This analogue with sys module, closes #20
37
38   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
39   Author: Bas van der Vlies
40
41 * More robust make/configure/setup.py
42   Reported by:
43        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
44        - Shunjie Lau <eijnuhs add gmail dot com>
45   Author: Bas van der Vlies
46
47
48 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
49   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
50   Author: Bas van der Vlies
51   
52=========== 4.0.2
53  * Quick fix so we do not get any undefined symbols. Forgot to
54    add the torque library when we compile the package.
55    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
56    Author: Bas van der Vlies
57
58=========== 4.0.1
59  * Fixed a bug in setting the right library path  for compiling.
60    Reported by: Chad Vizino ( vizino add psc dot edu )
61    Author: Bas van der Vlies
62   
63=========== 4.0.0
64  * pbs.py added some new functions for 2.4 version:
65    - pbs_fbserver(void);
66    - pbs_get_server_list(void);
67    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
68    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
69    - pbs_checkpointjob(int connect, char *job_id, char *extend);
70    - log_ext(int,char *,char *,int);
71    - log_init(char *, char *);
72    - log_remove_old(char *,unsigned long);
73
74    Note: version 2.1.X is still supported
75
76  * autotools environment
77    - Added support for finding pbs-config (borrowed for mpiexec source)
78    - Detecting which version of torque is installed
79    - only support installations that have pbs-config installed.
80    - Removed old code for installations that do not a have pbs-config
81      installed.
82
83  * PBSQuery.py
84    - There was a bug in the new data interface (Resource_List for jobs)
85      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
86      Applied by with some minor changes: Bas van der Vlies
87
88    - The new data structure is the default. You can switch back to the old
89      one with:
90            p = PBSQuery()
91            p.old_data_structure()
92
93      Author: Bas van der Vlies
94
95   * examples/ha_server.py
96     An example how to use the High Availabilty torque functions
97     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
98
99=========== 3.6.0
100  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
101    Reported by: Dennis Stam
102    Fixed by: Bas van der Vlies
103
104=========== 3.5.0
105  * PBSQuery
106        The class functions of node, job and queue support old and new data
107        structure.
108
109        Changed the behaviour of the new data stucture, We can use it as
110        dictionary and as class attribute, this is equivalent, eg:
111          - print node['np'] and print node.np
112
113        for a node we parse the 'status' line and split on '=' char, You now can
114        use these statements, eg
115          - print node.status.arch     (node['status'].arch or node['status']['arch'])
116          - print node.status.nsession
117
118        for a job we parse the 'Variable_List' line and split on '=' char, You now can
119        use the statements, eg:
120          - print job.Variable_List.PBS_O_WORKDIR
121          - print job.Variable_List.PBS_O_HOME
122
123        For more info see examples/new_interface.py
124
125    Author: Bas van der Vlies
126
127 * new_rack_pbsmon.py
128        Rewrite to new data structure and automatically determine how many nodes
129        and racks cluster has and skip printing of empty racks (default), use -w/--wide
130        for old behaviour.
131
132   Author: Bas van der Vlies
133
134
135=========== 3.2.0
136        - PBSQuery:
137          New data structure. In the old structure it is a dictionary
138          with a value and the value is a string. This is changed
139          that dictionary values are now of type list or dictionary depends
140          on the value of the keyword, eg for a node:
141            - np = 2:
142              * node['np'] = [ '2' ] 
143
144            - properties = cores2, mem4gb, parallel
145              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
146
147            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
148              * node['status']['arch'] = [ 'x86_64' ]
149              * node['status']['sessions'] = [ '222599' ]
150              * ...
151
152         The data structure is activated by the function:
153          * new_data_structure()
154
155        In a future release it will be come the default.
156        example:
157                p = PBSQuery()
158                p.new_data_structure()
159
160                nodes = p.getnodes()
161                print nodes.np, nodes['np']
162
163        Author: Bas van der Vlies
164
165        - PBSQuery
166          For old and new data structure we now can use another syntax:
167           * node['np'] and node.np are equivalent
168
169          This new syntax works for all keywords.
170
171        Author: Bas van der Vlies
172
173        - Added iter object for job, node, queue and server objects, eg:
174          node = p.getnode('gb-r1n1')
175          print node.name
176          for attrib in node:
177                print '%\t%s = %s' %(attrib, node[attrib])
178        Author: Bas van der Vlies
179
180        - fixed an error in getnode, getqueue and getjob, return empty
181          dictionary if not found
182        Author: Bas van der Vlies
183       
184        - New build system for rpm packages, make -f Makefile.rpm
185          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
186
187        Applied: Bas van der Vlies
188
189=========== 3.0.1
190        - PBSQuery
191                * Removed a testing code line
192                Reported By: sam @ vpac dot org
193                Fixed by: Bas van der Vlies
194
195=========== 3.0.0
196        - PBSQuery
197                * Cleaned up some old lines of code which didn't do anything.
198                * Changed the returnvalue of getqueue, getnode and getjob. Now
199                  they return an instance of the object, instead of a dict.
200                  This is API change so we bumped the version to 3.0.0
201                * Now the getnode-function accepts the short name as well as
202                  the long name as an argument.
203
204                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
205
206        - pbs.py, _pbs.so, PBSQuery.py
207                * Fixed a memory leak when used in daemon mode
208                Author: Bas van der Vlies <basv@sara.nl>
209
210=========== 2.9.8
211        - pbs_python.spec file patch added libdir and python defines so it
212          will build for RHEL5.1/CentOS 5.1
213          by        : Michael Sternberg <sternberg add anl dot gov>
214          applied by: Bas van der Vlies
215
216        - Updated the header files for pbs_python. So it can use some new
217          defines variables/defines, eg:
218                * ATTR_NODE_status
219                * ATTR_NODE_note
220                * ...
221          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
222
223        - Two new functions to log.h:
224                log_roll(int size);
225                log_size(void);
226          Added by: Bas van der Vlies
227
228        - We now have proper destructors (free memory) in SWIG for attrl and
229          attropl attributes.
230          Added by: Bas van der Vlies
231
232        - PBSQuery
233                * Added getnodes_with_property() function. This will get all nodes with
234                  a certain property, eg:
235                   - get all nodes with property 'cores8' and return all attributes:
236                      getnodes_with_property('cores8')
237               
238                   - get all nodes with property 'cores2' and only return the
239                     attributes 'state' and 'properties':
240                      attrl = [ 'state', 'properties' ]
241                      getnodes_with_property('cores2', attrl)
242               
243                     Only for torque version > 2.1.0
244             
245                Added by: Bas van der Vlies
246
247                * Added to class node the function
248                 - get_jobs(self, unique=None)
249                   Returns a list of the currently running job-id('s) on the node.
250                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
251
252                * Added get_server_name() function.
253                  This will return the PBS-server's name.
254                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
255
256                * Added to class _PBSobject the function
257                  - uniq(self, list)
258                    Filters out unique items of a list.
259                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
260
261                * Added to class job the function
262                  - get_nodes(self, unique=None)
263                    Returns a list of the nodes which run the job.
264                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
265
266=========== Version 2.9.4
267        -  Function pbs.pbs_geterrmsg does not exits any more is
268           replaced by pbs.error():
269                errno, text = pbs.error()
270                print errno, text
271           Reported by: pk at q-leap dot com
272           Fixed by   : Bas van der Vlies
273
274        - PBSQuery.py
275           * Added get_version() for server class, returns the version of
276             the batch server
277             Added by    : Bas van der Vlies
278
279           * Added get_nodes() for job class, returns a list of nodes on
280             which the job is run
281             Added by    : Bas van der Vlies
282
283           * PBSQuery.py rewrite make use of UserDict module.
284             Suggested by: Ramon Bastiaans
285             Fixed by    : Bas van der Vlies
286
287=========== Version 2.9.2
288        - The function pbs_rescquery() did not work. Fixed it
289        - The function avail(), did not work fixed it
290        - The functions pbs_rescreserve and pbs_rescrelease did not work
291          fixed it
292
293        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
294          nodes was wrong and state down has topmost priority
295
296=========== Version 2.9.0
297        - Added support for new torque build system 2.1.0
298        - Added new_rack_pbsmon.py
299          Authors: Walter de Jong & Bas van der Vlies
300
301=========== Version 2.8.2
302
303Configure has a new option:
304        --with-pbsdir=PATH
305
306If set it will use PATH specified as argument to
307find the pbs/torque libraries. If unset it will use
308some useful defaults to find the libaries.
309Suggested by: Troy Baer
310
311PBSQuery.py:
312        - If we can not make a connection with the server
313          then raise an exception and let the programmer
314          decides what to do.
315
316pbs_python.spec:
317        - It can now handle mode bits
318          Author: Martin Pels from SARA
319       
320=========== Version 2.8.0
321
322setup.py:
323        - Added /usr/lib to PBS_LIB_DIR
324
325pbs.py:
326        - Added new function "pbs.error()". This function checks if
327          an error has occured with a pbs function, eg:
328            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
329            error_number, err_txt = pbs.error()
330            if error_number:
331                print error_number, err_txt
332
333          prints the following message if script A.tsk does not exists:
334            15042 (qsub) cannot access script file
335
336pbs_python.spec:
337        - to make rpm packages from the source package, initial version
338        Author: Martin Pels from SARA
339
340README:
341        - Now requires python 2.1 and higher
342        - Explain how to build DEBIAN package
343
344=========== Version 2.7.10
345
346PBSQuery.py:
347        Fixed an error. You always got all resources from eg nodes
348        even when you supplied an attribute list where you only
349        requested the 'state' of the node.
350
351        Updated the inline python documentation
352
353=========== Version 2.7.9
354
355Fixed an error the previous version was in DEBUG mode. So
356you get an lot undesired output on the screen.
357
358debian package now also generated a package with the write
359version info.
360
361=========== Version 2.7.8
362
363Setup.py:
364        Can also handle ROCKS installations + the version
365        info is now the same as pbs.py
366        Thanks to: Roy Dragseth
367
368=========== Version 2.7.7
369
370PBSQuery module:
371  It can now be used in daemon programs. The pbs_server closes
372  the connection after an certain amount of time. This is fixed,
373  before every query there is an new connection and if the query
374  is finished the connection is closed.
375 
376=========== Version 2.7.6
377
378Now setup.py can also handle old Oscar Installations.
379Thanks To: Robin Humble
380
381Fixed an print statement fix in new_interface.py:
382Thanks To: Robin Humble
383
384Made some improvements for PBSQuery module:
385        - Updated the documentation
386        - Added the has_key() function to all
387          PBSObjects. So that the behaviour is similar to
388          an dictionary
389          Suggested By: Ramon Bastiaans
390       
391=========== Version 2.7.5
392 Added PBSQuery module.  This module requires pbs.py and it a simple module
393 for querying the pbs server. The documentation is in the module. Use for
394 example ipython to read it.
395
396 Written By: Roy Dragseth
397             Bas van der Vlies
398
399 eg: ( see also examples/new_interface.py)
400
401 from PBSQuery import PBSQuery
402 p = PBSQuery()
403
404 jobs = p.getjobs()
405 for name, job in jobs.items()
406        print job
407
408=========== Version 2.7.4
409 New versions fixes an bug in function pbs.pbs_statnode().
410 Reported by: Keith Poirier
411
412 Configure support:
413 Contributed By: Yaroslav Halchenko
414
415 Debian package support:
416 Contributed By: Yaroslav Halchenko
417
418 Added Support for Debuging the interface, You must edit pbs_wrap.c
419 and search for SARA_DEBUG:
420        #define SARA_DEBUG 1
421 Implemented by: Ramon Bastiaans
422
423 examples/pbsmon.py:
424        - Fixed an error when regex fails to determine node number,
425          no status was displayed
426        - It will now display and 'j' if the node is free for the batch
427          system and a job runs on the node (SMP-systems).
428 Changed by: Bas van der Vlies
429
430 examples/rack_pbsmon.py:
431        - An pbsmon that display node info per rack
432        - Edit the rack_pbsmon.py to adjust the values. It requires
433          that hostname contains rack and node id's,
434          eg: gb-r<number>n<number>
435 Contributed by: Walter de Jong
436
437=========== Version 2.7.3
438 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
439 Resource and QUEing manager).
440
441 The interface support OPENPBS and TORQUE.
442
443=========== Version 2.7.2
444 Some minor changes to include files. Now the Scalable PBS keywords
445 are also supported ( server attribute names: node_ping_rate and
446 node_check_rate). This does not interfere with the openpbs software.
447
448 pbsmon.py could not handle an one node cluster. Fixed it.
449
450=========== Version 2.7
451 Forgot to wrap the pbs_statfree() function. So we could not free
452 allocated memory from functions that return 'struct batch_status *'
453 like pbs_statjob().
454
455 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
456 Daniel Olson
457
458=========== Version 2.6.1
459 Setup.py now checks if all openpbs libraries are installed to compile the
460 package.
461
462=========== Version 2.6
463
464 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
465 function. This bug prevented to pass 'struct attrl' variables to this
466 function. Thanks to Evelyn Shiu for reporting this bug.
467
468
469=========== Version 2.5
470 Added the OpenPbs logging functions (log.h and liblog.a). There
471 is an example in the examples directory: logpbs.py
472
473 Fixed a bug in examples/resmom_info.py used the default port for
474 pbs_resmon if getservbyname fails.
475
476 Fixed a bug in resmom code. If the user has no permissions to query
477 the pbs_mom daemon a empty string is returned. Now we check for
478 the empty string. So we do not get a python exception.
479
480 Fixed a bug in setup.py forgot the exit statement when we did not
481 find the pbs libraries.
482
483=========== Version 2.3
484 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
485
486 Removed some obsolete code.
487
488 As for now we make use of the distutils module. So the package
489 is automatically compiled and installed in proper directories, usage:
490    python setup.py install
491 This requires that the package 'distutils' must be installed. As for
492 version 1.6 and higher it is in the distribution.
493
494 Added a function that prints the pbs python interface version. So we
495 can check if we have the right interface version:
496 import pbs
497 print pbs.version()
498 
499
500=========== Version 2.2
501
502In this release we can query the pbs_mom daemon with the aid of
503resource management functions:
504   pbs.openrm()
505   pbs.closerm()
506   pbs.downrm()
507   pbs.configrm()
508   pbs.addreq()
509   pbs.allreq()
510   pbs.flushreq()
511   pbs.activereq()
512   pbs.fullresp()
513   pbs.getreq()
514
515There is also a python function:
516   pbs.get_mom_values(id, [list]):
517     id   - connection id of pbs.openrm(node-name, port-number)
518     list - Is a optional paramter. A list of resource keywords
519
520   If 'list' is not supplied then this function will get the values
521   for the standard resource keywords, eg uname, loadave, .. + 'arch'
522   depended keywords. The arch feature is only implemented for linux,
523   but it can easily be ported to other oses.
524
525   if 'list' is supplied then we only fetch values for the keywords
526   that are in the list.
527
528   The function returns a dictonary. The keys are the resource keywords.
529
530   See 'resmom_info.py' for a example of this new feature.
531
532=========== Version 2.0
533
534The previous version was a simple interface above the PBS C API LIB.  This
535interface has gone a major changes. The most noticeable change is  that
536the functions accept/return Python lists instead of C-lists.  So you can now
537use standard Python syntax for manipulating Python lists. If you are familar
538with the Python syntax then you will appreciate this new interface.
539
540NOTE:
541  This interface is NOT compatible with the old one
542
543Here are some guidelines to convert your code to the new one:
544
545The constructors for the struct has gone a major change:
546  old code:
547    temp = pbs.new_attrl();
548    attrl_p = pbs.attrlPtr(temp)
549    attrl_p.name = 'state'
550
551  new_code:
552     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
553     attr_l[0].name = 'state'
554
555
556The pbs_stat functions returns Python lists instead of C-lists. There is
557NO next field anymore:
558  old_code:
559    temp = pbs.new_attrl();
560    attrl_p = pbs.attrlPtr(temp)
561    attrl_p.this = 'NULL'
562
563    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
564
565    while nodes.this != 'NULL':
566      print nodes.name
567
568      node_attrl = nodes.attribs
569      while node_attrl.this != 'NULL':
570        print '\t', node_attrl.name, '=', node_attrl.value
571        node_attrl = node_attrl.next
572
573    nodes = nodes.next
574
575
576  new_code:
577    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
578    for node in nodes:
579      print node.name
580      for attrib in node.attribs:
581        print '\t', attrib.name, '=', attrib.value
582
583Another advantage is you can use the print statement to show the
584connect of attrl and attropl classes:
585   attr_l = pbs.new_attrl(2)
586   attr_l[0].name = 'bas'
587   attr_l[0].value = 'van der Vlies'
588
589   print  attr_l[0]
590   >> (bas,,van der Vlies)
591   
592I hope these examples illustrate the changes. If you specify a wrong type
593for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.