source: trunk/CHANGES @ 223

Last change on this file since 223 was 223, checked in by bas, 14 years ago

src/PBSQuery.py

  • fixed an error in the new data structure, thanks to Mark Roberts
  • new data structure is the default

CHANGES:

  • preparing fro 4.0.0 release
  • Property svn:keywords set to Id
File size: 16.1 KB
Line 
1=========== 4.0.0
2  * pbs.py added some new functions for 2.4 version:
3    - pbs_fbserver(void);
4    - pbs_get_server_list(void);
5    - pbs_sigjobasync(int connect, char *job_id, char *signal, char *extend);
6    - pbs_alterjob_async(int connect, char *job_id, struct attrl *ATTRL, char *extend);
7    - pbs_checkpointjob(int connect, char *job_id, char *extend);
8    - log_ext(int,char *,char *,int);
9    - log_init(char *, char *);
10    - log_remove_old(char *,unsigned long);
11
12    Note: version 2.1.X is still supported
13
14  * autotools environment
15    - Added support for finding pbs-config (borrowed for mpiexec source)
16    - Detecting which version of torque is installed
17    - only support installations that have pbs-config installed.
18    - Removed old code for installations that do not a have pbs-config
19      installed.
20
21  * PBSQuery.py
22    - There was a bug in the new data interface (Resource_List for jobs)
23      Reported & Patch by: Mark Roberts ( mark at gingergeeks dot co dot uk)
24      Applied by with some minor changes: Bas van der Vlies
25
26    - The new data structure is the default. You can switch back to the old
27      one with:
28            p = PBSQuery()
29            p.old_data_structure()
30
31      Author: Bas van der Vlies
32
33=========== 3.6.0
34  * pbs, new generated pbs_wrap.c, pbs.py fixes a bug in pbs_runjob()
35    Reported by: Dennis Stam
36    Fixed by: Bas van der Vlies
37
38=========== 3.5.0
39  * PBSQuery
40        The class functions of node, job and queue support old and new data
41        structure.
42
43        Changed the behaviour of the new data stucture, We can use it as
44        dictionary and as class attribute, this is equivalent, eg:
45          - print node['np'] and print node.np
46
47        for a node we parse the 'status' line and split on '=' char, You now can
48        use these statements, eg
49          - print node.status.arch     (node['status'].arch or node['status']['arch'])
50          - print node.status.nsession
51
52        for a job we parse the 'Variable_List' line and split on '=' char, You now can
53        use the statements, eg:
54          - print job.Variable_List.PBS_O_WORKDIR
55          - print job.Variable_List.PBS_O_HOME
56
57        For more info see examples/new_interface.py
58
59    Author: Bas van der Vlies
60
61 * new_rack_pbsmon.py
62        Rewrite to new data structure and automatically determine how many nodes
63        and racks cluster has and skip printing of empty racks (default), use -w/--wide
64        for old behaviour.
65
66   Author: Bas van der Vlies
67
68
69=========== 3.2.0
70        - PBSQuery:
71          New data structure. In the old structure it is a dictionary
72          with a value and the value is a string. This is changed
73          that dictionary values are now of type list or dictionary depends
74          on the value of the keyword, eg for a node:
75            - np = 2:
76              * node['np'] = [ '2' ] 
77
78            - properties = cores2, mem4gb, parallel
79              * node['properties'] = [ 'cores2', 'mem4gb', 'parallel' ]
80
81            - status = arch=x86_64,sessions=22599,,size=70627864kb, ...
82              * node['status']['arch'] = [ 'x86_64' ]
83              * node['status']['sessions'] = [ '222599' ]
84              * ...
85
86         The data structure is activated by the function:
87          * new_data_structure()
88
89        In a future release it will be come the default.
90        example:
91                p = PBSQuery()
92                p.new_data_structure()
93
94                nodes = p.getnodes()
95                print nodes.np, nodes['np']
96
97        Author: Bas van der Vlies
98
99        - PBSQuery
100          For old and new data structure we now can use another syntax:
101           * node['np'] and node.np are equivalent
102
103          This new syntax works for all keywords.
104
105        Author: Bas van der Vlies
106
107        - Added iter object for job, node, queue and server objects, eg:
108          node = p.getnode('gb-r1n1')
109          print node.name
110          for attrib in node:
111                print '%\t%s = %s' %(attrib, node[attrib])
112        Author: Bas van der Vlies
113
114        - fixed an error in getnode, getqueue and getjob, return empty
115          dictionary if not found
116        Author: Bas van der Vlies
117       
118        - New build system for rpm packages, make -f Makefile.rpm
119          Author: Michel Jouvin <jouvin add lal dot in2p3 dot fr>
120
121        Applied: Bas van der Vlies
122
123=========== 3.0.1
124        - PBSQuery
125                * Removed a testing code line
126                Reported By: sam @ vpac dot org
127                Fixed by: Bas van der Vlies
128
129=========== 3.0.0
130        - PBSQuery
131                * Cleaned up some old lines of code which didn't do anything.
132                * Changed the returnvalue of getqueue, getnode and getjob. Now
133                  they return an instance of the object, instead of a dict.
134                  This is API change so we bumped the version to 3.0.0
135                * Now the getnode-function accepts the short name as well as
136                  the long name as an argument.
137
138                Author: Sil Westerveld <Sil.Westerveld@sara.nl>
139
140        - pbs.py, _pbs.so, PBSQuery.py
141                * Fixed a memory leak when used in daemon mode
142                Author: Bas van der Vlies <basv@sara.nl>
143
144=========== 2.9.8
145        - pbs_python.spec file patch added libdir and python defines so it
146          will build for RHEL5.1/CentOS 5.1
147          by        : Michael Sternberg <sternberg add anl dot gov>
148          applied by: Bas van der Vlies
149
150        - Updated the header files for pbs_python. So it can use some new
151          defines variables/defines, eg:
152                * ATTR_NODE_status
153                * ATTR_NODE_note
154                * ...
155          Added by: Dennis Stam <dennis.stam@sara.nl>, Bas van der Vlies
156
157        - Two new functions to log.h:
158                log_roll(int size);
159                log_size(void);
160          Added by: Bas van der Vlies
161
162        - We now have proper destructors (free memory) in SWIG for attrl and
163          attropl attributes.
164          Added by: Bas van der Vlies
165
166        - PBSQuery
167                * Added getnodes_with_property() function. This will get all nodes with
168                  a certain property, eg:
169                   - get all nodes with property 'cores8' and return all attributes:
170                      getnodes_with_property('cores8')
171               
172                   - get all nodes with property 'cores2' and only return the
173                     attributes 'state' and 'properties':
174                      attrl = [ 'state', 'properties' ]
175                      getnodes_with_property('cores2', attrl)
176               
177                     Only for torque version > 2.1.0
178             
179                Added by: Bas van der Vlies
180
181                * Added to class node the function
182                 - get_jobs(self, unique=None)
183                   Returns a list of the currently running job-id('s) on the node.
184                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
185
186                * Added get_server_name() function.
187                  This will return the PBS-server's name.
188                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
189
190                * Added to class _PBSobject the function
191                  - uniq(self, list)
192                    Filters out unique items of a list.
193                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
194
195                * Added to class job the function
196                  - get_nodes(self, unique=None)
197                    Returns a list of the nodes which run the job.
198                Added by: Sil Westerveld <Sil.Westerveld@sara.nl>
199
200=========== Version 2.9.4
201        -  Function pbs.pbs_geterrmsg does not exits any more is
202           replaced by pbs.error():
203                errno, text = pbs.error()
204                print errno, text
205           Reported by: pk at q-leap dot com
206           Fixed by   : Bas van der Vlies
207
208        - PBSQuery.py
209           * Added get_version() for server class, returns the version of
210             the batch server
211             Added by    : Bas van der Vlies
212
213           * Added get_nodes() for job class, returns a list of nodes on
214             which the job is run
215             Added by    : Bas van der Vlies
216
217           * PBSQuery.py rewrite make use of UserDict module.
218             Suggested by: Ramon Bastiaans
219             Fixed by    : Bas van der Vlies
220
221=========== Version 2.9.2
222        - The function pbs_rescquery() did not work. Fixed it
223        - The function avail(), did not work fixed it
224        - The functions pbs_rescreserve and pbs_rescrelease did not work
225          fixed it
226
227        - Fixed a bug in new_rack_pbsmon.py. Counting of serial
228          nodes was wrong and state down has topmost priority
229
230=========== Version 2.9.0
231        - Added support for new torque build system 2.1.0
232        - Added new_rack_pbsmon.py
233          Authors: Walter de Jong & Bas van der Vlies
234
235=========== Version 2.8.2
236
237Configure has a new option:
238        --with-pbsdir=PATH
239
240If set it will use PATH specified as argument to
241find the pbs/torque libraries. If unset it will use
242some useful defaults to find the libaries.
243Suggested by: Troy Baer
244
245PBSQuery.py:
246        - If we can not make a connection with the server
247          then raise an exception and let the programmer
248          decides what to do.
249
250pbs_python.spec:
251        - It can now handle mode bits
252          Author: Martin Pels from SARA
253       
254=========== Version 2.8.0
255
256setup.py:
257        - Added /usr/lib to PBS_LIB_DIR
258
259pbs.py:
260        - Added new function "pbs.error()". This function checks if
261          an error has occured with a pbs function, eg:
262            task_id = pbs.pbs_submit(c, attrl, "A.tsk", 'NULL', 'NULL')
263            error_number, err_txt = pbs.error()
264            if error_number:
265                print error_number, err_txt
266
267          prints the following message if script A.tsk does not exists:
268            15042 (qsub) cannot access script file
269
270pbs_python.spec:
271        - to make rpm packages from the source package, initial version
272        Author: Martin Pels from SARA
273
274README:
275        - Now requires python 2.1 and higher
276        - Explain how to build DEBIAN package
277
278=========== Version 2.7.10
279
280PBSQuery.py:
281        Fixed an error. You always got all resources from eg nodes
282        even when you supplied an attribute list where you only
283        requested the 'state' of the node.
284
285        Updated the inline python documentation
286
287=========== Version 2.7.9
288
289Fixed an error the previous version was in DEBUG mode. So
290you get an lot undesired output on the screen.
291
292debian package now also generated a package with the write
293version info.
294
295=========== Version 2.7.8
296
297Setup.py:
298        Can also handle ROCKS installations + the version
299        info is now the same as pbs.py
300        Thanks to: Roy Dragseth
301
302=========== Version 2.7.7
303
304PBSQuery module:
305  It can now be used in daemon programs. The pbs_server closes
306  the connection after an certain amount of time. This is fixed,
307  before every query there is an new connection and if the query
308  is finished the connection is closed.
309 
310=========== Version 2.7.6
311
312Now setup.py can also handle old Oscar Installations.
313Thanks To: Robin Humble
314
315Fixed an print statement fix in new_interface.py:
316Thanks To: Robin Humble
317
318Made some improvements for PBSQuery module:
319        - Updated the documentation
320        - Added the has_key() function to all
321          PBSObjects. So that the behaviour is similar to
322          an dictionary
323          Suggested By: Ramon Bastiaans
324       
325=========== Version 2.7.5
326 Added PBSQuery module.  This module requires pbs.py and it a simple module
327 for querying the pbs server. The documentation is in the module. Use for
328 example ipython to read it.
329
330 Written By: Roy Dragseth
331             Bas van der Vlies
332
333 eg: ( see also examples/new_interface.py)
334
335 from PBSQuery import PBSQuery
336 p = PBSQuery()
337
338 jobs = p.getjobs()
339 for name, job in jobs.items()
340        print job
341
342=========== Version 2.7.4
343 New versions fixes an bug in function pbs.pbs_statnode().
344 Reported by: Keith Poirier
345
346 Configure support:
347 Contributed By: Yaroslav Halchenko
348
349 Debian package support:
350 Contributed By: Yaroslav Halchenko
351
352 Added Support for Debuging the interface, You must edit pbs_wrap.c
353 and search for SARA_DEBUG:
354        #define SARA_DEBUG 1
355 Implemented by: Ramon Bastiaans
356
357 examples/pbsmon.py:
358        - Fixed an error when regex fails to determine node number,
359          no status was displayed
360        - It will now display and 'j' if the node is free for the batch
361          system and a job runs on the node (SMP-systems).
362 Changed by: Bas van der Vlies
363
364 examples/rack_pbsmon.py:
365        - An pbsmon that display node info per rack
366        - Edit the rack_pbsmon.py to adjust the values. It requires
367          that hostname contains rack and node id's,
368          eg: gb-r<number>n<number>
369 Contributed by: Walter de Jong
370
371=========== Version 2.7.3
372 The name SPBS is changed to the new name TORQUE (Tera-scale Open-source
373 Resource and QUEing manager).
374
375 The interface support OPENPBS and TORQUE.
376
377=========== Version 2.7.2
378 Some minor changes to include files. Now the Scalable PBS keywords
379 are also supported ( server attribute names: node_ping_rate and
380 node_check_rate). This does not interfere with the openpbs software.
381
382 pbsmon.py could not handle an one node cluster. Fixed it.
383
384=========== Version 2.7
385 Forgot to wrap the pbs_statfree() function. So we could not free
386 allocated memory from functions that return 'struct batch_status *'
387 like pbs_statjob().
388
389 pbsmon.py can now handle 2 and 3 digit hostnames. Patch supplied by
390 Daniel Olson
391
392=========== Version 2.6.1
393 Setup.py now checks if all openpbs libraries are installed to compile the
394 package.
395
396=========== Version 2.6
397
398 Fixed a bug in the pbs python module. Forgot to map the pbs_statjob()
399 function. This bug prevented to pass 'struct attrl' variables to this
400 function. Thanks to Evelyn Shiu for reporting this bug.
401
402
403=========== Version 2.5
404 Added the OpenPbs logging functions (log.h and liblog.a). There
405 is an example in the examples directory: logpbs.py
406
407 Fixed a bug in examples/resmom_info.py used the default port for
408 pbs_resmon if getservbyname fails.
409
410 Fixed a bug in resmom code. If the user has no permissions to query
411 the pbs_mom daemon a empty string is returned. Now we check for
412 the empty string. So we do not get a python exception.
413
414 Fixed a bug in setup.py forgot the exit statement when we did not
415 find the pbs libraries.
416
417=========== Version 2.3
418 Bugfix in the pbsnodes-a.py. Forgot to import the sys module.
419
420 Removed some obsolete code.
421
422 As for now we make use of the distutils module. So the package
423 is automatically compiled and installed in proper directories, usage:
424    python setup.py install
425 This requires that the package 'distutils' must be installed. As for
426 version 1.6 and higher it is in the distribution.
427
428 Added a function that prints the pbs python interface version. So we
429 can check if we have the right interface version:
430 import pbs
431 print pbs.version()
432 
433
434=========== Version 2.2
435
436In this release we can query the pbs_mom daemon with the aid of
437resource management functions:
438   pbs.openrm()
439   pbs.closerm()
440   pbs.downrm()
441   pbs.configrm()
442   pbs.addreq()
443   pbs.allreq()
444   pbs.flushreq()
445   pbs.activereq()
446   pbs.fullresp()
447   pbs.getreq()
448
449There is also a python function:
450   pbs.get_mom_values(id, [list]):
451     id   - connection id of pbs.openrm(node-name, port-number)
452     list - Is a optional paramter. A list of resource keywords
453
454   If 'list' is not supplied then this function will get the values
455   for the standard resource keywords, eg uname, loadave, .. + 'arch'
456   depended keywords. The arch feature is only implemented for linux,
457   but it can easily be ported to other oses.
458
459   if 'list' is supplied then we only fetch values for the keywords
460   that are in the list.
461
462   The function returns a dictonary. The keys are the resource keywords.
463
464   See 'resmom_info.py' for a example of this new feature.
465
466=========== Version 2.0
467
468The previous version was a simple interface above the PBS C API LIB.  This
469interface has gone a major changes. The most noticeable change is  that
470the functions accept/return Python lists instead of C-lists.  So you can now
471use standard Python syntax for manipulating Python lists. If you are familar
472with the Python syntax then you will appreciate this new interface.
473
474NOTE:
475  This interface is NOT compatible with the old one
476
477Here are some guidelines to convert your code to the new one:
478
479The constructors for the struct has gone a major change:
480  old code:
481    temp = pbs.new_attrl();
482    attrl_p = pbs.attrlPtr(temp)
483    attrl_p.name = 'state'
484
485  new_code:
486     attr_l = pbs.new_attrl(1)   // Creates a list of attrl structs length 1
487     attr_l[0].name = 'state'
488
489
490The pbs_stat functions returns Python lists instead of C-lists. There is
491NO next field anymore:
492  old_code:
493    temp = pbs.new_attrl();
494    attrl_p = pbs.attrlPtr(temp)
495    attrl_p.this = 'NULL'
496
497    nodes = pbs.pbs_statnode(con, "", attrl_p, "NULL")
498
499    while nodes.this != 'NULL':
500      print nodes.name
501
502      node_attrl = nodes.attribs
503      while node_attrl.this != 'NULL':
504        print '\t', node_attrl.name, '=', node_attrl.value
505        node_attrl = node_attrl.next
506
507    nodes = nodes.next
508
509
510  new_code:
511    nodes = pbs.pbs_statnode(con, "", "NULL", "NULL")
512    for node in nodes:
513      print node.name
514      for attrib in node.attribs:
515        print '\t', attrib.name, '=', attrib.value
516
517Another advantage is you can use the print statement to show the
518connect of attrl and attropl classes:
519   attr_l = pbs.new_attrl(2)
520   attr_l[0].name = 'bas'
521   attr_l[0].value = 'van der Vlies'
522
523   print  attr_l[0]
524   >> (bas,,van der Vlies)
525   
526I hope these examples illustrate the changes. If you specify a wrong type
527for a function then function wil raise a Python exception.
Note: See TracBrowser for help on using the repository browser.