OSNews

gentoo is probably the worst advertisement for python. On anything but a very modern fast machine the portage operations are painfully, painfully slow and IMHO show exactly why scripting languages shouldn't be used too ambitiously for things. They just don't scale well.

gentoo is probably the worst advertisement for python. On anything but a very modern fast machine the portage operations are painfully, painfully slow and IMHO show exactly why scripting languages shouldn't be used too ambitiously for things. They just don't scale well.

I don't think python is the real issue. Portage used to be very fast. Over the years though it has gotten bloated and inefficient. Extending it has been difficult and not without loss of efficiency. It really is time for a rewrite. People seem to be leaning towards a C++ solution but I would rather have Python so we can easily extend it and add new modules.

I think it is exacly the point of bnolsen

When portage was getting more complex, python wasn't the language to do the job anymore. A static typing, compiled language would be better. more error checking and compiled validation.

If you tried to use python application for more than scripting (music player, IDE, etc.) you'll see that as it become more complex, side effect make application unstable (they crash alot). Debuging start to make development no more productive than using compiled language (without pointer).

I develop mainly in python, but sometimes one need to face reality.

When portage was getting more complex, python wasn't the language to do the job anymore. A static typing, compiled language would be better. more error checking and compiled validation.

I don't agree. I don't think it was complexity in general but instead portage just wasn't designed well enough to be extended to the degree it has been. This isn't a limitation of Python but a limitation of portage's initial design.

If you tried to use python application for more than scripting (music player, IDE, etc.) you'll see that as it become more complex, side effect make application unstable (they crash alot). Debuging start to make development no more productive than using compiled language (without pointer).

I use PIDA for Python development and it works very well for me. The latest version is very slick and hasn't crashed on me yet. I've also experimented with a few Python music players, like Quod Libet, which haven't caused me any stability issues although I find the interface lacking, which is hardly a complaint against Python.

"When portage was getting more complex, python wasn't the language to do the job anymore. A static typing, compiled language would be better. more error checking and compiled validation."

Wait, we are talking about a package manager here, aren't we? Why on earth would a package manager require the extra effort of developing in a compiled language (C/C++) just to get a performance boost? I mean... how can a package manager be performance critical?

I don't know portage, but I would bet the problem is in the design and not in the language... yum used to be _brutally_ slow too, and it is much faster now, even though it keeps being Python. Not downloading tons of crap every time it is invoked helps a lot...

And it's just a package manager...

The problems with portage are more due to design than language. If you where to re-write portage in C using exactly the same design, algorithms and architecture it's currently using, it wouldn't be much faster. Conversly I you where to redesign it from the ground up, and still use python, it would be significantly faster.

the first time i ran across python as system scripts was in a fresh gentoo install. it crashed each time i tried to proceed....

You must have had a foobarred installation. I had an issue with python on Gentoo exactly once in the 4 years I have been using Gentoo. Otherwise it is a solid language and other than the slow syncs and slow dependency checking it works fine for portage.

Edited 2007-09-15 14:52

While I agree that Python is not a replacement for the shell, your post show the shell's problem: first there is not one shell, but several and the 'script shells' depends on a bunch of tools, all of which have variants depending on the version, OS, etc.

Most script shells are *brittle*, use them on another shell, OS, on file with spaces or weird characters and they break.

What I would like to see is a bunch of tools written in Ruby/Python which would send struct/hash through the pipe: you would have the flexibility of shell and the robustness induced by passing structures/hash around instead of 'hard to parse' text (and if you have to take into account files with weird characters in them, it's really not so easy).

That was pretty much the motivation for creating Perl back in the 80s. The situation has improved quite a lot since then. The end of the Unix wars brought a good deal of industry consolidation and the standardization of all those tools. Nowadays it's quite feasible to create shell scripts devoid of this kind of problems.

Even the shell is standardized. POSIX 1003.2 made /bin/sh a KornShell lookalike rather than the historical Bourne shell. That's the only shell you should use for scripts.

But the one true reason to prefer shell scripts is that way you can leverage your knowledge of the command line. Conversely, writing shell scripts will improve your skills in all the standard Unix utilities. Once you start using Perl, Python, etc. you'll have to learn two ways of doing the same thing: One way to do it interactively and one to script it.

Using bash/ksh/nawk/grep etc. is fine for some things. And of course, that's where it should be used.

We have a guy at work who has pretty much the same attitude as you and he insists on doing EVERYTHING in ksh/nawk. He wrote things like XML parsers in nawk (not fully functional, of course, but they almost work for what he needs it for)

In the end, he manages to create slow, error-
prone incomprehensible stuff that would have been fairly simple, fast and easier to make correct in for example python.

So hold on to your bash, but just don't take it too far.

Heh, reminds me of a place I used to work. Their reporting system was kinda like this:

* The server gets a request for report X.
* Server grabs list of environment variables for report X from the DB
* Server sets the environment variables with values passed in the request.
* Server spawns ksh with the appropriate reporting script.
* Shell script uses IO redirection to send SQL to Oracle
* Shell script parses the data and stores it to a temp file.
* The server reads the temp file and sends it back to the client.

The great part is, the guy who actually "designed" that part, left for some other job and nobody else there knew Korn shell.

I think that's exactly what the articles suggests. There's no implying that Python should replace Bash, but complement it for those scripts that are too much for Bash. If anything, it attempts to replace Perl, not Bash.

The great thing about using shell scripts to their maximum capacity is that there are no modules. With perl and python, the suggestion for common use involves a module, which may or may _not_ be on the system.

You will find that once you go down the path of attempting to install a module for an interpreted language, your systems can become out of sync with one another. Not so with shell scripts. A Bourne shell function works as well on one system as it certainly will another.

Even when programming to capture missing modules, have you found that exception use of import (python) or require (perl) involves a lot of alarm bells and whistles. And you do this so that you catch the exception to be able to begin the installation of the module dependencies and module rather than actually getting on with better things to do?

>>I perfer scripts that run with native shells that do not require an interpreter (sh/ksh/csh) so I can use them in any systemz<<

But, other than sh, you can't always count on those shells being on any particular system. Especially if it's a windows box.

I used to have the same philosophy, until I started writing 10 line elegant Python scripts, that do more than I could do with 100 lines of clunky shell scripts.

Try Python with easygui, it's amazing. Within minutes you can be writing fairly powerful scripts with a gui interface. The learning curve is nothing.

I have a few Windows boxes and they host for the most part well behaved applications, our interaction with them is minimal. Our challenges are working with Solaris and Linux using CLI only (no X allowed), but thanks for the tip though! It reminds me of using TCL/Tk.

I recently rediscovered Rebol (http://www.rebol.com) and giving it a good look over. It's not only able to run on multiple OSes but apps built with it also run on multiple OSes (a ton of them) without recoding.

It's as easy to work with as Python (or arguably moreso) for the beginner (but I'm still evaluating it from that standpoint). It also features built-in GUI (amazingly simple to implement and use) - so there's little need for external help.

The EasyGUI info you provided is great help - thanks! I'm trying to make a decision on a cross-platform scripting language after many years on Win-machines using single-OS only script languages (batches, 4dos, kixtart, Autohotkey, etc..). I'm still leaning heavily toward Python because I know just how easy to use and powerful this language is but this "new" Rebol kid is making it tough:-)

Actually "Many old skool sys admins" use C, and wonder why people like program run by other programs

mzscheme makes a better scripting language than you might think.

After using Python for a few years I decided to check out Ruby, and I did look back. Not that Ruby is not a good tool, but I just can't shake that feeling of unease around anything with a syntax that reminds me of perl.

at least as i know, perl is not default in a FreeBSD install.

I believe perl was at one time included as part of the FBSD base installation, but it was removed as it began to bloat (and some other issues).

It's now available as a port.

The last time perl was included by default was 4.x - much of the system scripts that used perl were rewritten in bourne shell..

Subjectively, I disagree with that style of thinking. As a system administrator, you should know how to work in a minimal environment, yes. But to suggest that you shouldn't avail yourself of a good tool because some vendor doesn't put it in a default install sounds either too narrowly focused on barebones without acknowledging that not everyone wants such a thing, or stagnant. I want neither.

Use the best tool for the job. In text processing, perl beats python.

In python, a regexp is a second class object. In perl, a regexp is a first class object. Anyone with proficiency in *both* languages will agree with you.

I love python, but it doesn't make sense for managing "Unix" systems. If the title of the article was, "Why You Should Adopt Python to Manage Linux Systems", I would agree.

You can't always install something as large as python due to business processes or technical requirements. When your company has been managing systems a similar way for 10 years, they might not like to change. (Not reflective of my current job, but true for my previous one.)

Edited 2007-09-15 02:30

"In python, a regexp is a second class object. In perl, a regexp is a first class object. Anyone with proficiency in *both* languages will agree with you."

Well, if you consider regexp build into the syntax a better solution, then I guess perl is ahead. I don't find Pythons regexp hard to use and I like the design where you don't cram things like regexp into the language, but keep it in a module.
It will make me type a little bit more, but it also means I usually understand my code better than in perl anyway.

Having said that, I'm not using Python for system administration much. The simple things that I need to do can be done in shell scripts just as well as in Python. The times I use Python is when things should run automatically and a "Permission denied" printed to stderr just isn't enough error handling.

Python has exception handling and perl doesn't. You have just hit on an area that Python tromps perl about a millionfold. Python's OO is also not a pain to deal with as it is truly integrated into the language.

That said, Linux systems administration is mostly just dealing with text files and text streams. For that alone, perl will always beat python.

Yes, it's true. But also is true that some machines doesn't haven't some programs (any program or for a multiple reasons), because customer instructions, environment necessities.
Many years ago back, I used the "joe" editor, until a day said my boss to me; "you would have to be learning to use "vi", because you are going it to have in any UNIX with which you work, whereas the "joe" editor, or a cc (gcc) environment no.

No it isn't. The idea that Perl is some sort of standard is what drives me (and I suspect quite a few other sysadmins and developers) up the wall.

Dude, that's so lame...

So, install it.



Seriously, how many times per year do you use a PERL one-liner from the command line? 4? Maybe 5 times?



This is not a safe practice in any interpreted language. Better to write those sections using a compiled language, take your choice.



... and about a dozen other languages and tools. Its syntax is certainly the most expansive syntax of any programming language in history. You might think that's a good thing, but try picking up a piece of code someone else wrote in Perl, and maintaining it.

We have been installing Python on all our Unix boxes for about 5 years now, and I think it was a great move.

ksh93, Python, C. That's all you need for every Unix administrative programming project. And the C is only needed when you need to wrapper a system call into a Python function. They're used in descending order of frequency: ksh93 - 70%, Python - 25%, C - 5%.

bash,sed/awk,perl,C fits my bill :-)

"bash,sed/awk,perl,C fits my bill :-)"

Sidenote: Don't declare bash as long as you're not using special bash features that are not present in standard sh. sh is the most basal common sense in UNIX scripting. bas isn't (allthough it's very famous in Linux).

Furthermore, I agree to the command set above for most administration tasks that need to be automated. sed and awk are very powerfull, especially when you want to implement "small" tasks with few commands. On the other hand, I like using Python and Ruby for several tasks, but I never thought seriously about using them intensively for administrative tasks. I'll go out and correct my attitude. :-)

I tried to take that route when converting a major system from Unix to Linux. The Unix system used Posix shell, and I thought the best approach would be to use bash with the Posix compliant switch turned on. Unfortunately, that was too limiting. There were several key features that were not present in the bash sh mode but did have an analogue in plain bash. Plus the fact that most Linux stuff was set to run bash by default, it meant I was continually having to be very sure what I was mixing. I finally bit the bullet and converted every script to bash. I guess I could have taken another approach, but this was more direct and made me happier in the long run.

(Not to mention that there are all sorts of other incompatibilities between Unix and Linux utilities that had to taken care of.)

I used C when required for speed or library calls, but no other scripting language. I did not want to introduce another tool unless absolutely necesary. Shell script can be verbose, but it can also be very nicely formatted and readable if you take the time.

If had to do it over again without the time constraints, maybe I would have chosen a scripting language to make it more portable in the future. But time always forces a less than ideal solution!

"Sidenote: Don't declare bash as long as you're not using special bash features that are not present in standard sh. sh is the most basal common sense in UNIX scripting. bas isn't (allthough it's very famous in Linux)."

That doesn't work if you are developing your scripts in Linux (where "/bin/sh" is usually just a symlink to "/bin/bash"). You end up thinking your scripts can run in the standard bourne shell when they actually don't.

If you want portability, just use "/bin/ksh". It's mostly compatible with bash (so you can use advanced features without having to relearn syntax) and it's available by default in most other unixes. In Linux, if it isn't installed by default it's usually an "apt-get install" or "yum install" away, so not a big deal.

"Seriously, how many times per year do you use a PERL one-liner from the command line? 4? Maybe 5 times?"

And the fact that 99.9% of perl one-liners result from people not knowing how to use "sed" and "grep".

Actually I find 99.9% of perl one-liners result from perl nerds trying to show off, and prove that it can be done.

Why is he getting modded down? It's a good Python tutorial.

I'm modded down by the OSNews system by default because I'm considered a troll (even if most of the times my input is valuable). Telling the truth or not thinking like the mainstream is not a good thing.

Granted I'm a 'nix guy, but the idea of ClusterRdesktop (for windows) would be really cool as an equivalent to something like clusterssh on Linux.

Real men, (Or woman..) do things manually ^_^

Umm, no. We women are smart enough to leave such things to "real men" ;P

Please stop. Comments like that, which are all too common, make the Ruby community seem like a bunch of teenagers.

Rarely do I see comments like "Python Rulezzzzz!" in Ruby-related threads. But, for whatever reason, some Ruby fans just can't resist getting childish when the topic of Python comes up.

I *prefer* Python. It's a great language. But I choose not to trumpet the fact in Ruby-oriented threads, because I feel that it would be in poor taste, and would not be constructive.

Now you're just being silly.

"Now you're just being silly. "

That's not silly. That would be silly:

C:/MICROS~1/SYSADMIN/QBASIC.EXE ruhletzzzz!!! :-)

"Wake me when there's a pysh, a python shell."

Okay then. Wake up!
http://pysh.sourceforge.net/
http://sourceforge.net/projects/pyshell/

But I agree with the pysh author, it probably isn't the right way to go.

The reason not every unix system has Python, or even perl installed is that the tools that are available are doing a pretty good job for most simple automation tasks.

Just as the reason Microsoft made PowerShell is that the tools on that platform didn't really do a good job at all kind of simple automation tasks.

http://ipython.scipy.org/moin/

IPython is a multiplatform, Free Software project (BSD licensed) that offers:

* An