source: trunk/CHANGES @ 249

Last change on this file since 249 was 249, checked in by bas, 14 years ago

updated CHANGES
  • Property svn:keywords set to Id
File size: 18.7 KB
Line 
1=========== 4.1.3
2 * Updated LICENSE file to LGPLV3. Thanks to Justin Bronder
3  (jsbronder add gentoo dot org) for the information and
4  explanation.
5
6 * examples/pbsmon.py. Patched so it also works for torque 2.5
7   Author: Stijn De Weirdt stijn dot deweirdt at ugent dot be
8   Applied by: Bas van der Vlies
9
10 * setup.py.in changed determine which torque version is installed
11   so we also support snapshots.
12   Author: Mark Roberts ( mark at gingergeeks dot co dot uk)
13   Applied by: Bas van der Vlies
14
15 * PBSQuery:
16   - get_nodes function for jobs now works for old and new datastructure and
17     gives correct results if unique=True
18     Author: Bas van der Vlies
19
20 * examples/new_rack_pbsmon.py:
21     - Better specification for node expression
22     - Added AdvanceParser module, Author: Dennis Stam
23     - added a new option -l/--list to list the state of a nodes and which job
24       is running on it, eg:
25             new_rack_pbsmon.py - l gb-r[1-3]n[3-6] gb-r4[n1-20]
26
27   Author: Dennis Stam & Bas van der Vlies
28   Applied by: Bas van der Vlies
29
30=========== 4.1.2
31 * examples/new_rack_pbsmon.py now uses the new PBSQuery data
32   structure.
33   Author: Bas van der Vlies
34
35 * PBSQuery:
36     - fixed and error in get_jobs function for nodes. Did not
37       support new data structure and uniq did not work.
38       Authors: Shunjie Lau <eijnuhs at gmail dot com> and Bas van der Vlies
39
40=========== 4.1.0
41 * Delete function pbs.version() replaced by
42    * pbs.version
43        This is a string variable
44    * pbs.version_info
45        Is a tuple variable. So we can use it in scripts
46
47   This analogue with sys module, closes #20
48
49   Requested by: Dennis Stam ( Dennis dot Stam add sara dot nl )
50   Author: Bas van der Vlies
51
52 * More robust make/configure/setup.py
53   Reported by:
54        - Saidi, Yacine <saidi add linmpi dot mpg dot de>
55        - Shunjie Lau <eijnuhs add gmail dot com>
56   Author: Bas van der Vlies
57
58
59 * Only use the pbs_wrap.c and pbs.py 2.4 version for torque version 2.4.7 and higher.
60   Reported by: Saidi, Yacine <saidi add linmpi dot mpg dot de>
61   Author: Bas van der Vlies
62   
63=========== 4.0.2
64  * Quick fix so we do not get any undefined symbols. Forgot to
65    add the torque library when we compile the package.
66    Reported by: Shunjie Lau ( eijnuhs add gmail dot com)
67    Author: Bas van der Vlies
68
69=========== 4.0.1
70  * Fixed a bug in setting the right library path  for compiling.
71    Reported by: Chad Vizino ( vizino add psc dot edu )
72    Author: Bas van der Vlies
73   
74=========== 4.0.0
75  * pbs.py added some new functions for 2.4 version:
76    - pbs_fbserver(void);
77    - pbs_get_server_list(void);
78    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
79    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
80    - pbs_checkpointjob(int connect, char *job_id, char *extend);
81    - log_ext(int,char *,char *,int);
82    - log_init(char *, char *);
83    - log_remove_old(char *,unsigned long);
84
85    Note: version 2.1.X is still supported
86
87  * autotools environment
88    - Added support for finding pbs-config (borrowed for mpiexec source)
89    - Detecting which version of torque is installed
90    - only support installations that have pbs-config installed.
91    - Removed old code for installations that do not a have pbs-config
92      installed.
93
94  * PBSQuery.py
95    - There was a bug in the new data interface (Resource_List for jobs)
96      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
97      Applied by with some minor changes: Bas van der Vlies
98
99    - The new data structure is the default. You can switch back to the old
100      one with:
101            p = PBSQuery()
102            p.old_data_structure()
103
104      Author: Bas van der Vlies
105
106   * examples/ha_server.py
107     An example how to use the High Availabilty torque functions
108     Author: Bas van der Vlies & Mark Roberts ( mark at gingergeeks dot co dot uk )
109
110=========== 3.6.0
111  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
112    Reported by: Dennis Stam
113    Fixed by: Bas van der Vlies
114
115=========== 3.5.0
116  * PBSQuery
117        The class functions of node, job and queue support old and new data
118        structure.
119
120        Changed the behaviour of the new data stucture, We can use it as
121        dictionary and as class attribute, this is equivalent, eg:
122          - print node['np'] and print node.np
123
124        for a node we parse the 'status' line and split on '=' char, You now can
125        use these statements, eg
126          - print node.status.arch     (node['status'].arch or node['status']['arch'])
127          - print node.status.nsession
128
129        for a job we parse the 'Variable_List' line and split on '=' char, You now can
130        use the statements, eg:
131          - print job.Variable_List.PBS_O_WORKDIR
132          - print job.Variable_List.PBS_O_HOME
133
134        For more info see examples/new_interface.py
135
136    Author: Bas van der Vlies
137
138 * new_rack_pbsmon.py
139        Rewrite to new data structure and automatically determine how many nodes
140        and racks cluster has and skip printing of empty racks (default), use -w/--wide
141        for old behaviour.
142
143   Author: Bas van der Vlies
144
145
146=========== 3.2.0
147        - PBSQuery:
148          New data structure. In the old structure it is a dictionary
149          with a value and the value is a string. This is changed
150          that dictionary values are now of type list or dictionary depends
151          on the value of the keyword, eg for a node:
152            - np = 2:
153              * node['np'] = [ '2' ] 
154
155            - properties = cores2, mem4gb, parallel
156              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
157
158            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
159              * node['status']['arch'] = [ 'x86_64' ]
160              * node['status']['sessions'] = [ '222599' ]
161              * ...
162
163         The data structure is activated by the function:
164          * new_data_structure()
165
166        In a future release it will be come the default.
167        example:
168                p = PBSQuery()
169                p.new_data_structure()
170
171                nodes = p.getnodes()
172                print nodes.np, nodes['np']
173
174        Author: Bas van der Vlies
175
176        - PBSQuery
177          For old and new data structure we now can use another syntax:
178           * node['np'] and node.np are equivalent
179
180          This new syntax works for all keywords.
181
182        Author: Bas van der Vlies
183
184        - Added iter object for job, node, queue and server objects, eg:
185          node = p.getnode('gb-r1n1')
186          print node.name
187          for attrib in node:
188                print '%\t%s = %s' %(attrib, node[attrib])
189        Author: Bas van der Vlies
190
191        - fixed an error in getnode, getqueue and getjob, return empty
192          dictionary if not found
193        Author: Bas van der Vlies
194       
195        - New build system for rpm packages, make -f Makefile.rpm
196          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
197
198        Applied: Bas van der Vlies
199
200=========== 3.0.1
201        - PBSQuery
202                * Removed a testing code line
203                Reported By: sam @ vpac dot org
204                Fixed by: Bas van der Vlies
205
206=========== 3.0.0
207        - PBSQuery
208                * Cleaned up some old lines of code which didn't do anything.
209                * Changed the returnvalue of getqueue, getnode and getjob. Now
210                  they return an instance of the object, instead of a dict.
211                  This is API change so we bumped the version to 3.0.0
212                * Now the getnode-function accepts the short name as well as
213                  the long name as an argument.
214
215                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
216
217        - pbs.py, _pbs.so, PBSQuery.py
218                * Fixed a memory leak when used in daemon mode
219                Author: Bas van der Vlies <basv@sara.nl>
220
221=========== 2.9.8
222        - pbs_python.spec file patch added libdir and python defines so it
223          will build for RHEL5.1/CentOS 5.1
224          by        : Michael Sternberg <sternberg add anl dot gov>
225          applied by: Bas van der Vlies
226
227        - Updated the header files for pbs_python. So it can use some new
228          defines variables/defines, eg:
229                * ATTR_NODE_status
230                * ATTR_NODE_note
231                * ...
232          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
233
234        - Two new functions to log.h:
235                log_roll(int size);
236                log_size(void);
237          Added by: Bas van der Vlies
238
239        - We now have proper destructors (free memory) in SWIG for attrl and
240          attropl attributes.
241          Added by: Bas van der Vlies
242
243        - PBSQuery
244                * Added getnodes_with_property() function. This will get all nodes with
245                  a certain property, eg:
246                   - get all nodes with property 'cores8' and return all attributes:
247                      getnodes_with_property('cores8')
248               
249                   - get all nodes with property 'cores2' and only return the
250                     attributes 'state' and 'properties':
251                      attrl = [ 'state', 'properties' ]
252                      getnodes_with_property('cores2', attrl)
253               
254                     Only for torque version > 2.1.0
255             
256                Added by: Bas van der Vlies
257
258                * Added to class node the function
259                 - get_jobs(self, unique=None)
260                   Returns a list of the currently running job-id('s) on the node.
261                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
262
263                * Added get_server_name() function.
264                  This will return the PBS-server's name.
265                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
266
267                * Added to class _PBSobject the function
268                  - uniq(self, list)
269                    Filters out unique items of a list.
270                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
271
272                * Added to class job the function
273                  - get_nodes(self, unique=None)
274                    Returns a list of the nodes which run the job.
275                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
276
277=========== Version 2.9.4
278        -  Function pbs.pbs_geterrmsg does not exits any more is
279           replaced by pbs.error():
280                errno, text = pbs.error()
281                print errno, text
282           Reported by: pk at q-leap dot com
283           Fixed by   : Bas van der Vlies
284
285        - PBSQuery.py
286           * Added get_version() for server class, returns the version of
287             the batch server
288             Added by    : Bas van der Vlies
289
290           * Added get_nodes() for job class, returns a list of nodes on
291             which the job is run
292             Added by    : Bas van der Vlies
293
294           * PBSQuery.py rewrite make use of UserDict module.
295             Suggested by: Ramon Bastiaans
296             Fixed by    : Bas van der Vlies
297
298=========== Version 2.9.2
299        - The function pbs_rescquery() did not work. Fixed it
300        - The function avail(), did not work fixed it
301        - The functions pbs_rescreserve and pbs_rescrelease did not work
302          fixed it
303
304        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
305          nodes was wrong and state down has topmost priority
306
307=========== Version 2.9.0
308        - Added support for new torque build system 2.1.0
309        - Added new_rack_pbsmon.py
310          Authors: Walter de Jong & Bas van der Vlies
311
312=========== Version 2.8.2
313
314Configure has a new option:
315        --with-pbsdir=PATH
316
317If set it will use PATH specified as argument to
318find the pbs/torque libraries. If unset it will use
319some useful defaults to find the libaries.
320Suggested by: Troy Baer
321
322PBSQuery.py:
323        - If we can not make a connection with the server
324          then raise an exception and let the programmer
325          decides what to do.
326
327pbs_python.spec:
328        - It can now handle mode bits
329          Author: Martin Pels from SARA
330       
331=========== Version 2.8.0
332
333setup.py:
334        - Added /usr/lib to PBS_LIB_DIR
335
336pbs.py:
337        - Added new function "pbs.error()". This function checks if
338          an error has occured with a pbs function, eg:
339            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
340            error_number, err_txt = pbs.error()
341            if error_number:
342                print error_number, err_txt
343
344          prints the following message if script A.tsk does not exists:
345            15042 (qsub) cannot access script file
346
347pbs_python.spec:
348        - to make rpm packages from the source package, initial version
349        Author: Martin Pels from SARA
350
351README:
352        - Now requires python 2.1 and higher
353        - Explain how to build DEBIAN package
354
355=========== Version 2.7.10
356
357PBSQuery.py:
358        Fixed an error. You always got all resources from eg nodes
359        even when you supplied an attribute list where you only
360        requested the 'state' of the node.
361
362        Updated the inline python documentation
363
364=========== Version 2.7.9
365
366Fixed an error the previous version was in DEBUG mode. So
367you get an lot undesired output on the screen.
368
369debian package now also generated a package with the write
370version info.
371
372=========== Version 2.7.8
373
374Setup.py:
375        Can also handle ROCKS installations + the version
376        info is now the same as pbs.py
377        Thanks to: Roy Dragseth
378
379=========== Version 2.7.7
380
381PBSQuery module:
382  It can now be used in daemon programs. The pbs_server closes
383  the connection after an certain amount of time. This is fixed,
384  before every query there is an new connection and if the query
385  is finished the connection is closed.
386 
387=========== Version 2.7.6
388
389Now setup.py can also handle old Oscar Installations.
390Thanks To: Robin Humble
391
392Fixed an print statement fix in new_interface.py:
393Thanks To: Robin Humble
394
395Made some improvements for PBSQuery module:
396        - Updated the documentation
397        - Added the has_key() function to all
398          PBSObjects. So that the behaviour is similar to
399          an dictionary
400          Suggested By: Ramon Bastiaans
401       
402=========== Version 2.7.5
403 Added PBSQuery module.  This module requires pbs.py and it a simple module
404 for querying the pbs server. The documentation is in the module. Use for
405 example ipython to read it.
406
407 Written By: Roy Dragseth
408             Bas van der Vlies
409
410 eg: ( see also examples/new_interface.py)
411
412 from PBSQuery import PBSQuery
413 p = PBSQuery()
414
415 jobs = p.getjobs()
416 for name, job in jobs.items()
417        print job
418
419=========== Version 2.7.4
420 New versions fixes an bug in function pbs.pbs_statnode().
421 Reported by: Keith Poirier
422
423 Configure support:
424 Contributed By: Yaroslav Halchenko
425
426 Debian package support:
427 Contributed By: Yaroslav Halchenko
428
429 Added Support for Debuging the interface, You must edit pbs_wrap.c
430 and search for SARA_DEBUG:
431        #define SARA_DEBUG 1
432 Implemented by: Ramon Bastiaans
433
434 examples/pbsmon.py:
435        - Fixed an error when regex fails to determine node number,
436          no status was displayed
437        - It will now display and 'j' if the node is free for the batch
438          system and a job runs on the node (SMP-systems).
439 Changed by: Bas van der Vlies
440
441 examples/rack_pbsmon.py:
442        - An pbsmon that display node info per rack
443        - Edit the rack_pbsmon.py to adjust the values. It requires
444          that hostname contains rack and node id's,
445          eg: gb-r<number>n<number>
446 Contributed by: Walter de Jong
447
448=========== Version 2.7.3
449 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
450 Resource and QUEing manager).
451
452 The interface support OPENPBS and TORQUE.
453
454=========== Version 2.7.2
455 Some minor changes to include files. Now the Scalable PBS keywords
456 are also supported ( server attribute names: node_ping_rate and
457 node_check_rate). This does not interfere with the openpbs software.
458
459 pbsmon.py could not handle an one node cluster. Fixed it.
460
461=========== Version 2.7
462 Forgot to wrap the pbs_statfree() function. So we could not free
463 allocated memory from functions that return 'struct batch_status *'
464 like pbs_statjob().
465
466 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
467 Daniel Olson
468
469=========== Version 2.6.1
470 Setup.py now checks if all openpbs libraries are installed to compile the
471 package.
472
473=========== Version 2.6
474
475 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
476 function. This bug prevented to pass 'struct attrl' variables to this
477 function. Thanks to Evelyn Shiu for reporting this bug.
478
479
480=========== Version 2.5
481 Added the OpenPbs logging functions (log.h and liblog.a). There
482 is an example in the examples directory: logpbs.py
483
484 Fixed a bug in examples/resmom_info.py used the default port for
485 pbs_resmon if getservbyname fails.
486
487 Fixed a bug in resmom code. If the user has no permissions to query
488 the pbs_mom daemon a empty string is returned. Now we check for
489 the empty string. So we do not get a python exception.
490
491 Fixed a bug in setup.py forgot the exit statement when we did not
492 find the pbs libraries.
493
494=========== Version 2.3
495 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
496
497 Removed some obsolete code.
498
499 As for now we make use of the distutils module. So the package
500 is automatically compiled and installed in proper directories, usage:
501    python setup.py install
502 This requires that the package 'distutils' must be installed. As for
503 version 1.6 and higher it is in the distribution.
504
505 Added a function that prints the pbs python interface version. So we
506 can check if we have the right interface version:
507 import pbs
508 print pbs.version()
509 
510
511=========== Version 2.2
512
513In this release we can query the pbs_mom daemon with the aid of
514resource management functions:
515   pbs.openrm()
516   pbs.closerm()
517   pbs.downrm()
518   pbs.configrm()
519   pbs.addreq()
520   pbs.allreq()
521   pbs.flushreq()
522   pbs.activereq()
523   pbs.fullresp()
524   pbs.getreq()
525
526There is also a python function:
527   pbs.get_mom_values(id, [list]):
528     id   - connection id of pbs.openrm(node-name, port-number)
529     list - Is a optional paramter. A list of resource keywords
530
531   If 'list' is not supplied then this function will get the values
532   for the standard resource keywords, eg uname, loadave, .. + 'arch'
533   depended keywords. The arch feature is only implemented for linux,
534   but it can easily be ported to other oses.
535
536   if 'list' is supplied then we only fetch values for the keywords
537   that are in the list.
538
539   The function returns a dictonary. The keys are the resource keywords.
540
541   See 'resmom_info.py' for a example of this new feature.
542
543=========== Version 2.0
544
545The previous version was a simple interface above the PBS C API LIB.  This
546interface has gone a major changes. The most noticeable change is  that
547the functions accept/return Python lists instead of C-lists.  So you can now
548use standard Python syntax for manipulating Python lists. If you are familar
549with the Python syntax then you will appreciate this new interface.
550
551NOTE:
552  This interface is NOT compatible with the old one
553
554Here are some guidelines to convert your code to the new one:
555
556The constructors for the struct has gone a major change:
557  old code:
558    temp = pbs.new_attrl();
559    attrl_p = pbs.attrlPtr(temp)
560    attrl_p.name = 'state'
561
562  new_code:
563     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
564     attr_l[0].name = 'state'
565
566
567The pbs_stat functions returns Python lists instead of C-lists. There is
568NO next field anymore:
569  old_code:
570    temp = pbs.new_attrl();
571    attrl_p = pbs.attrlPtr(temp)
572    attrl_p.this = 'NULL'
573
574    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
575
576    while nodes.this != 'NULL':
577      print nodes.name
578
579      node_attrl = nodes.attribs
580      while node_attrl.this != 'NULL':
581        print '\t', node_attrl.name, '=', node_attrl.value
582        node_attrl = node_attrl.next
583
584    nodes = nodes.next
585
586
587  new_code:
588    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
589    for node in nodes:
590      print node.name
591      for attrib in node.attribs:
592        print '\t', attrib.name, '=', attrib.value
593
594Another advantage is you can use the print statement to show the
595connect of attrl and attropl classes:
596   attr_l = pbs.new_attrl(2)
597   attr_l[0].name = 'bas'
598   attr_l[0].value = 'van der Vlies'
599
600   print  attr_l[0]
601   >> (bas,,van der Vlies)
602   
603I hope these examples illustrate the changes. If you specify a wrong type
604for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.