source: trunk/CHANGES @ 205

Last change on this file since 205 was 205, checked in by bas, 14 years ago

Preparing for new version 3.5

src/PBSQuery.py:

  • rewrite of dictionary code
  • Property svn:keywords set to Id
File size: 14.7 KB
Line 
1=========== 3.5.0
2  * PBSQuery
3        The class functions of node, job and queue support old and new data
4        structure.
5
6        Changed the behaviour of the new data stucture, We can use it as
7        dictionary and as class attribute, this is equivalent, eg:
8          - print node['np'] and print node.np
9
10        for a node we parse the 'status' line and split on '=' char, You now can
11        use these statements, eg
12          - print node.status.arch     (node['status'].arch or node['status']['arch'])
13          - print node.status.nsession
14
15        for a job we parse the 'Variable_List' line and split on '=' char, You now can
16        use the statements, eg:
17          - print job.Variable_List.PBS_O_WORKDIR
18          - print job.Variable_List.PBS_O_HOME
19
20        For more info see examples/new_interface.py
21
22    Author: Bas van der Vlies
23
24 * new_rack_pbsmon.py
25        Rewrite to new data structure and automatically determine how many nodes
26        and racks cluster has and skip printing of empty racks (default), use -w/--wide
27        for old behaviour.
28
29   Author: Bas van der Vlies
30
31
32=========== 3.2.0
33        - PBSQuery:
34          New data structure. In the old structure it is a dictionary
35          with a value and the value is a string. This is changed
36          that dictionary values are now of type list or dictionary depends
37          on the value of the keyword, eg for a node:
38            - np = 2:
39              * node['np'] = [ '2' ] 
40
41            - properties = cores2, mem4gb, parallel
42              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
43
44            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
45              * node['status']['arch'] = [ 'x86_64' ]
46              * node['status']['sessions'] = [ '222599' ]
47              * ...
48
49         The data structure is activated by the function:
50          * new_data_structure()
51
52        In a future release it will be come the default.
53        example:
54                p = PBSQuery()
55                p.new_data_structure()
56
57                nodes = p.getnodes()
58                print nodes.np, nodes['np']
59
60        Author: Bas van der Vlies
61
62        - PBSQuery
63          For old and new data structure we now can use another syntax:
64           * node['np'] and node.np are equivalent
65
66          This new syntax works for all keywords.
67
68        Author: Bas van der Vlies
69
70        - Added iter object for job, node, queue and server objects, eg:
71          node = p.getnode('gb-r1n1')
72          print node.name
73          for attrib in node:
74                print '%\t%s = %s' %(attrib, node[attrib])
75        Author: Bas van der Vlies
76
77        - fixed an error in getnode, getqueue and getjob, return empty
78          dictionary if not found
79        Author: Bas van der Vlies
80       
81        - New build system for rpm packages, make -f Makefile.rpm
82          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
83
84        Applied: Bas van der Vlies
85
86=========== 3.0.1
87        - PBSQuery
88                * Removed a testing code line
89                Reported By: sam @ vpac dot org
90                Fixed by: Bas van der Vlies
91
92=========== 3.0.0
93        - PBSQuery
94                * Cleaned up some old lines of code which didn't do anything.
95                * Changed the returnvalue of getqueue, getnode and getjob. Now
96                  they return an instance of the object, instead of a dict.
97                  This is API change so we bumped the version to 3.0.0
98                * Now the getnode-function accepts the short name as well as
99                  the long name as an argument.
100
101                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
102
103        - pbs.py, _pbs.so, PBSQuery.py
104                * Fixed a memory leak when used in daemon mode
105                Author: Bas van der Vlies <basv@sara.nl>
106
107=========== 2.9.8
108        - pbs_python.spec file patch added libdir and python defines so it
109          will build for RHEL5.1/CentOS 5.1
110          by        : Michael Sternberg <sternberg add anl dot gov>
111          applied by: Bas van der Vlies
112
113        - Updated the header files for pbs_python. So it can use some new
114          defines variables/defines, eg:
115                * ATTR_NODE_status
116                * ATTR_NODE_note
117                * ...
118          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
119
120        - Two new functions to log.h:
121                log_roll(int size);
122                log_size(void);
123          Added by: Bas van der Vlies
124
125        - We now have proper destructors (free memory) in SWIG for attrl and
126          attropl attributes.
127          Added by: Bas van der Vlies
128
129        - PBSQuery
130                * Added getnodes_with_property() function. This will get all nodes with
131                  a certain property, eg:
132                   - get all nodes with property 'cores8' and return all attributes:
133                      getnodes_with_property('cores8')
134               
135                   - get all nodes with property 'cores2' and only return the
136                     attributes 'state' and 'properties':
137                      attrl = [ 'state', 'properties' ]
138                      getnodes_with_property('cores2', attrl)
139               
140                     Only for torque version > 2.1.0
141             
142                Added by: Bas van der Vlies
143
144                * Added to class node the function
145                 - get_jobs(self, unique=None)
146                   Returns a list of the currently running job-id('s) on the node.
147                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
148
149                * Added get_server_name() function.
150                  This will return the PBS-server's name.
151                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
152
153                * Added to class _PBSobject the function
154                  - uniq(self, list)
155                    Filters out unique items of a list.
156                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
157
158                * Added to class job the function
159                  - get_nodes(self, unique=None)
160                    Returns a list of the nodes which run the job.
161                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
162
163=========== Version 2.9.4
164        -  Function pbs.pbs_geterrmsg does not exits any more is
165           replaced by pbs.error():
166                errno, text = pbs.error()
167                print errno, text
168           Reported by: pk at q-leap dot com
169           Fixed by   : Bas van der Vlies
170
171        - PBSQuery.py
172           * Added get_version() for server class, returns the version of
173             the batch server
174             Added by    : Bas van der Vlies
175
176           * Added get_nodes() for job class, returns a list of nodes on
177             which the job is run
178             Added by    : Bas van der Vlies
179
180           * PBSQuery.py rewrite make use of UserDict module.
181             Suggested by: Ramon Bastiaans
182             Fixed by    : Bas van der Vlies
183
184=========== Version 2.9.2
185        - The function pbs_rescquery() did not work. Fixed it
186        - The function avail(), did not work fixed it
187        - The functions pbs_rescreserve and pbs_rescrelease did not work
188          fixed it
189
190        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
191          nodes was wrong and state down has topmost priority
192
193=========== Version 2.9.0
194        - Added support for new torque build system 2.1.0
195        - Added new_rack_pbsmon.py
196          Authors: Walter de Jong & Bas van der Vlies
197
198=========== Version 2.8.2
199
200Configure has a new option:
201        --with-pbsdir=PATH
202
203If set it will use PATH specified as argument to
204find the pbs/torque libraries. If unset it will use
205some useful defaults to find the libaries.
206Suggested by: Troy Baer
207
208PBSQuery.py:
209        - If we can not make a connection with the server
210          then raise an exception and let the programmer
211          decides what to do.
212
213pbs_python.spec:
214        - It can now handle mode bits
215          Author: Martin Pels from SARA
216       
217=========== Version 2.8.0
218
219setup.py:
220        - Added /usr/lib to PBS_LIB_DIR
221
222pbs.py:
223        - Added new function "pbs.error()". This function checks if
224          an error has occured with a pbs function, eg:
225            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
226            error_number, err_txt = pbs.error()
227            if error_number:
228                print error_number, err_txt
229
230          prints the following message if script A.tsk does not exists:
231            15042 (qsub) cannot access script file
232
233pbs_python.spec:
234        - to make rpm packages from the source package, initial version
235        Author: Martin Pels from SARA
236
237README:
238        - Now requires python 2.1 and higher
239        - Explain how to build DEBIAN package
240
241=========== Version 2.7.10
242
243PBSQuery.py:
244        Fixed an error. You always got all resources from eg nodes
245        even when you supplied an attribute list where you only
246        requested the 'state' of the node.
247
248        Updated the inline python documentation
249
250=========== Version 2.7.9
251
252Fixed an error the previous version was in DEBUG mode. So
253you get an lot undesired output on the screen.
254
255debian package now also generated a package with the write
256version info.
257
258=========== Version 2.7.8
259
260Setup.py:
261        Can also handle ROCKS installations + the version
262        info is now the same as pbs.py
263        Thanks to: Roy Dragseth
264
265=========== Version 2.7.7
266
267PBSQuery module:
268  It can now be used in daemon programs. The pbs_server closes
269  the connection after an certain amount of time. This is fixed,
270  before every query there is an new connection and if the query
271  is finished the connection is closed.
272 
273=========== Version 2.7.6
274
275Now setup.py can also handle old Oscar Installations.
276Thanks To: Robin Humble
277
278Fixed an print statement fix in new_interface.py:
279Thanks To: Robin Humble
280
281Made some improvements for PBSQuery module:
282        - Updated the documentation
283        - Added the has_key() function to all
284          PBSObjects. So that the behaviour is similar to
285          an dictionary
286          Suggested By: Ramon Bastiaans
287       
288=========== Version 2.7.5
289 Added PBSQuery module.  This module requires pbs.py and it a simple module
290 for querying the pbs server. The documentation is in the module. Use for
291 example ipython to read it.
292
293 Written By: Roy Dragseth
294             Bas van der Vlies
295
296 eg: ( see also examples/new_interface.py)
297
298 from PBSQuery import PBSQuery
299 p = PBSQuery()
300
301 jobs = p.getjobs()
302 for name, job in jobs.items()
303        print job
304
305=========== Version 2.7.4
306 New versions fixes an bug in function pbs.pbs_statnode().
307 Reported by: Keith Poirier
308
309 Configure support:
310 Contributed By: Yaroslav Halchenko
311
312 Debian package support:
313 Contributed By: Yaroslav Halchenko
314
315 Added Support for Debuging the interface, You must edit pbs_wrap.c
316 and search for SARA_DEBUG:
317        #define SARA_DEBUG 1
318 Implemented by: Ramon Bastiaans
319
320 examples/pbsmon.py:
321        - Fixed an error when regex fails to determine node number,
322          no status was displayed
323        - It will now display and 'j' if the node is free for the batch
324          system and a job runs on the node (SMP-systems).
325 Changed by: Bas van der Vlies
326
327 examples/rack_pbsmon.py:
328        - An pbsmon that display node info per rack
329        - Edit the rack_pbsmon.py to adjust the values. It requires
330          that hostname contains rack and node id's,
331          eg: gb-r<number>n<number>
332 Contributed by: Walter de Jong
333
334=========== Version 2.7.3
335 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
336 Resource and QUEing manager).
337
338 The interface support OPENPBS and TORQUE.
339
340=========== Version 2.7.2
341 Some minor changes to include files. Now the Scalable PBS keywords
342 are also supported ( server attribute names: node_ping_rate and
343 node_check_rate). This does not interfere with the openpbs software.
344
345 pbsmon.py could not handle an one node cluster. Fixed it.
346
347=========== Version 2.7
348 Forgot to wrap the pbs_statfree() function. So we could not free
349 allocated memory from functions that return 'struct batch_status *'
350 like pbs_statjob().
351
352 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
353 Daniel Olson
354
355=========== Version 2.6.1
356 Setup.py now checks if all openpbs libraries are installed to compile the
357 package.
358
359=========== Version 2.6
360
361 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
362 function. This bug prevented to pass 'struct attrl' variables to this
363 function. Thanks to Evelyn Shiu for reporting this bug.
364
365
366=========== Version 2.5
367 Added the OpenPbs logging functions (log.h and liblog.a). There
368 is an example in the examples directory: logpbs.py
369
370 Fixed a bug in examples/resmom_info.py used the default port for
371 pbs_resmon if getservbyname fails.
372
373 Fixed a bug in resmom code. If the user has no permissions to query
374 the pbs_mom daemon a empty string is returned. Now we check for
375 the empty string. So we do not get a python exception.
376
377 Fixed a bug in setup.py forgot the exit statement when we did not
378 find the pbs libraries.
379
380=========== Version 2.3
381 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
382
383 Removed some obsolete code.
384
385 As for now we make use of the distutils module. So the package
386 is automatically compiled and installed in proper directories, usage:
387    python setup.py install
388 This requires that the package 'distutils' must be installed. As for
389 version 1.6 and higher it is in the distribution.
390
391 Added a function that prints the pbs python interface version. So we
392 can check if we have the right interface version:
393 import pbs
394 print pbs.version()
395 
396
397=========== Version 2.2
398
399In this release we can query the pbs_mom daemon with the aid of
400resource management functions:
401   pbs.openrm()
402   pbs.closerm()
403   pbs.downrm()
404   pbs.configrm()
405   pbs.addreq()
406   pbs.allreq()
407   pbs.flushreq()
408   pbs.activereq()
409   pbs.fullresp()
410   pbs.getreq()
411
412There is also a python function:
413   pbs.get_mom_values(id, [list]):
414     id   - connection id of pbs.openrm(node-name, port-number)
415     list - Is a optional paramter. A list of resource keywords
416
417   If 'list' is not supplied then this function will get the values
418   for the standard resource keywords, eg uname, loadave, .. + 'arch'
419   depended keywords. The arch feature is only implemented for linux,
420   but it can easily be ported to other oses.
421
422   if 'list' is supplied then we only fetch values for the keywords
423   that are in the list.
424
425   The function returns a dictonary. The keys are the resource keywords.
426
427   See 'resmom_info.py' for a example of this new feature.
428
429=========== Version 2.0
430
431The previous version was a simple interface above the PBS C API LIB.  This
432interface has gone a major changes. The most noticeable change is  that
433the functions accept/return Python lists instead of C-lists.  So you can now
434use standard Python syntax for manipulating Python lists. If you are familar
435with the Python syntax then you will appreciate this new interface.
436
437NOTE:
438  This interface is NOT compatible with the old one
439
440Here are some guidelines to convert your code to the new one:
441
442The constructors for the struct has gone a major change:
443  old code:
444    temp = pbs.new_attrl();
445    attrl_p = pbs.attrlPtr(temp)
446    attrl_p.name = 'state'
447
448  new_code:
449     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
450     attr_l[0].name = 'state'
451
452
453The pbs_stat functions returns Python lists instead of C-lists. There is
454NO next field anymore:
455  old_code:
456    temp = pbs.new_attrl();
457    attrl_p = pbs.attrlPtr(temp)
458    attrl_p.this = 'NULL'
459
460    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
461
462    while nodes.this != 'NULL':
463      print nodes.name
464
465      node_attrl = nodes.attribs
466      while node_attrl.this != 'NULL':
467        print '\t', node_attrl.name, '=', node_attrl.value
468        node_attrl = node_attrl.next
469
470    nodes = nodes.next
471
472
473  new_code:
474    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
475    for node in nodes:
476      print node.name
477      for attrib in node.attribs:
478        print '\t', attrib.name, '=', attrib.value
479
480Another advantage is you can use the print statement to show the
481connect of attrl and attropl classes:
482   attr_l = pbs.new_attrl(2)
483   attr_l[0].name = 'bas'
484   attr_l[0].value = 'van der Vlies'
485
486   print  attr_l[0]
487   >> (bas,,van der Vlies)
488   
489I hope these examples illustrate the changes. If you specify a wrong type
490for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.