Comments on: Generate Text From Templates, Scripts and CSV Data

By: Marco

Marco — Wed, 07 Oct 2009 15:57:26 +0000

Improved version. The original script breaks if there are white spaces in the fields.

#!/bin/bash
IMPORT=”./data/import-data.txt”
TEMPLATE=”./data/import-template.txt”

# for i in `cat ${IMPORT}`

cat $IMPORT | \
while read i

do
# echo $i # debug

VAR_HOSTNAME=`echo $i | awk -F, ‘{print $1}’`
VAR_IPADDR=`echo $i | awk -F, ‘{print $2}’`
VAR_NETMASK=`echo $i | awk -F, ‘{print $3}’`
VAR_OSPF_KEY=`echo $i | awk -F, ‘{print $4}’`
VAR_TACACS_KEY=`echo $i | awk -F, ‘{print $5}’`
cat $TEMPLATE | sed -e “s/VAR_HOSTNAME/$VAR_HOSTNAME/g” \
-e “s/VAR_IPADDR/$VAR_IPADDR/g” \
-e “s/VAR_NETMASK/$VAR_NETMASK/g” \
-e “s/VAR_OSPF_KEY/$VAR_OSPF_KEY/g” \
-e “s/VAR_TACACS_KEY/$VAR_TACACS_KEY/g” \
| tee ./output/”$VAR_HOSTNAME.txt” 1>/dev/null
done

By: gmendoza

gmendoza — Mon, 24 Aug 2009 15:56:42 +0000

1>file.txt sends output to a text file called “file.txt”.
1>>file.txt sends output to a text file called “file.txt”, but appends if it already exists.

2>file.txt sends errors to a text file called “file.txt”.
2>>file.txt sends errors to a text file called “file.txt”, but appends if it already exists.

Here’s a decent tutorial on the subject.
http://linux.about.com/library/bl/open/newbie/blnewbie3.4.7.htm

I’m really glad this was able to help you.

By: Laz

Laz — Mon, 24 Aug 2009 15:29:13 +0000

This has been extremely useful for both routers and switches. I had to configure about 45 devices, with this I was able to do it in about an hour after making the CSV file. I have one question though ( as I am a new to linux) what is “1>>./output/$VAR_HOSTNAME.txt” mean?

Thanks again!

By: gmendoza

gmendoza — Tue, 13 Jan 2009 05:31:16 +0000

@Herouth, Re: using tee…

Meh… that’s actually kind of funny. It was probably something left over from something else I was doing. It could easily be replaced with: 1>>./output/$VAR_HOSTNAME.txt

By: Herouth

Herouth — Mon, 12 Jan 2009 16:11:32 +0000

Oops, forgot the crucial part above: read -u 4

By: Herouth

Herouth — Mon, 12 Jan 2009 16:10:31 +0000

Here is how I would do the loop, which I think is more efficient (not invoking awk) – but requires that the import file does not have any empty fields:

exec 4<”$IMPORT”

while IFS=’,’ read VAR_HOSTNAME VAR_IPADDR VAR_NETMASK VAR_OSPF_KEY VAR_TACACS_KEY
do

# sed part remains unchanged

done

By the way, why are you using tee if you’re not doing anything with standard output?

By: Larry

Larry — Mon, 10 Mar 2008 18:23:58 +0000

Sweet scripts, not that it’s much use to me.

By: gmendoza

gmendoza — Sun, 09 Mar 2008 16:46:58 +0000

Hey there, Blacky. Thanks for contributing again.

The scenario I had in mind was purely for short-term time savings through bulk processing. The whole point of using a CSV is that it’s easy to import and parse thousands of records, as each iteration of the for-loop only has to process a single line to obtain the definitions separated by a delimiter. For this type of task, creating a CSV is easier to generate than cpp precompiler directives for subsitution, because the latter as you pointed out requires not only multiple lines for the definitions, but multiple files to serve as the import data.

However, what I really like about your suggestion is that if the intention is to maintain a version control system, individualizing the import data as a template of it’s own would be quite beneficial! Plus, you have the added benefit of simply being able to generate a single configuration from your cpp template, rather than having to create a new CSV for the job.

Now of course, I would still recommend using the method mentioned in the article to both generate and populate the cpp templates to get things started. But for continual maintenance of such text, your suggestion is certainly one that readers should look into.

Thanks again.

By: blacky

blacky — Sat, 08 Mar 2008 11:19:56 +0000

If the number of variables needed increases, the sed solution can
become quite unwieldy. Also, having different sets of defined
variables isn’t quite as easy. Finally, if you have many switches
(and many admins), it may be easier to have one file per switch
to facilitate the use of a version control system. To this end,
using the C preprocessor (or a similar macro system) may be
useful.

For example:

$ cat import-template.txt ! hostname HOSTNAME ! interface Vlan100 ip address IPADDR NETMASK ip ospf message-digest-key 1 md5 OSPF_KEY ! tacacs-server host 1.1.1.1 tacacs-server key TACACS_KEY !


$ cat switches/switch-01

#define HOSTNAME switch-01

#define IPADDR 10.1.1.1

#define NETMASK 255.255.255.0

#define OSPF_KEY ospf-key-1

#define TACACS_KEY tacacs-key-1

$ cpp -P -include switches/switch-01 import-template.txt

! hostname switch-01 ! interface Vlan100 ip address 10.1.1.1 255.255.255.0 ip ospf message-digest-key 1 md5 ospf-key-1 ! tacacs-server host 1.1.1.1 tacacs-server key tacacs-key-1 !