In this new release I have created a Manual page with some nice formatting. And expanded on some of the options now providing more meaningful information. The man page can be seen by typing:
man ./script
In the future this would be added to the PATH so that it can be seen from any location the same with the actual script.
As for the script I have expanded the help section to provide the information given in the man page.
Also I have automated the retrieval of the pidora kernel that I was previously unable to do it because I had no idea how to construct the url that provided me with the source RPM.
Some other notes:
For the Pidora source RPM it only looks for packages in the f19-rpfr Tag when changing to f20-rpfr this line would need to be modified.
The -b option now will take 3.11 instead of 3.11.y, using 3.11.y will cause errors. It is better anyways to use 3.11 just simpler.
There is no error checking so if the information provided is not right it will not finish successfully.
Still no name for the script .... Poor nameless script.
Since the scripts where getting too long I am instead linking to a file.
Man page:
https://drive.google.com/file/d/0B2G5T2vxJmLrUEhhZ1BrQ1JYWDA/edit?usp=sharing
Script:
https://drive.google.com/file/d/0B2G5T2vxJmLrTjJzcTNDeENEeDA/edit?usp=sharing
And I am off to Ecuador, it was great working on this. I will continue working on it when I return.
I plan on adding some error checking, some other useful options and cleaning up the the code more.
Friday, December 13, 2013
Thursday, December 5, 2013
Release 0.2.1
Various bugfixes
Added email option to report when a new kernel is building - could not test it because bell blocks port 25
-------------------------------------------------------------------------------------------------
#!/bin/bash
#Check if the file contain the commit long exists
#If it does not exist create the file
if [ ! -f ./branches ]; then
touch branches
fi
if [ ! -f ./long_commit ]; then
touch long_commit
fi
# Retrieve branches from github - Retrieving branches from ls-remote instead of wget
#wget -qO- https://github.com/raspberrypi/linux | cat | grep 'data-name="rpi' | cut -d '"' -f2 > ./branches
# Delete the patches branch - Used only when using the wget way
#sed -i '$ d' ./branches
# Storing git remote head values on a file
git ls-remote https://github.com/raspberrypi/linux.git > ls_remote
# Retrieving Branches from ls_remote
# The sort is only cheking for the decimal place if a version 4.x comes out this will stop working if 3.x versions
# remain when doing a git ls-remote
cat ls_remote | grep rpi | cut -d '/' -f3 | sort -t $'.' -n -k2,2 > ./branches
# Store latest branch in a variable
branch=$(tail -1 ./branches)
# Storing url for use with wget
url=$(echo "https://github.com/raspberrypi/linux/tree/$branch/kernel")
#wget -qO- $url > ./page_source
# Retrieving informacion from the git ls-remote command
#commit_date=$( cat ./page_source | grep 'committed <time class="js-relative-date"' | cut -d '"' -f6 | cut -d ' ' -f1 | cut -d '-' -f1-3 --output-delimiter='' )
commit_date=$( date +"%Y%m%d")
long_commit=$( cat ./ls_remote | grep $branch | cut -f1)
short_commit=$( echo $long_commit | cut -c 1-7)
stored_long_commit=$(cat ./long_commit)
# Finding out the %_topdir value to see where the rpmbuild directory is being created
topdir=$( rpm --eval '%{_topdir}')
# Find latest package submitted to rpfr18-updates-automated
koji_build=$( armv6-koji latest-pkg f18-rpfr-updates-automated raspberrypi-kernel | tail -1 | cut -d ' ' -f1)
# Safe to use koji URL
#koji_url=$( echo "http://japan.proximity.on.ca/kojifiles/packages/raspberrypi-kernel/3.11.6/4.20131023git10bc582.rpfr19/src/raspberrypi-kernel-3.11.6-4.20131023git10bc582.rpfr19.src.rpm" )
koji_url=$( echo "http://japan.proximity.on.ca/kojifiles/packages/raspberrypi-kernel/3.12.0/4.20131106git839f349.rpfr18/src/raspberrypi-kernel-3.12.0-4.20131106git839f349.rpfr18.src.rpm" )
# Usage function for the script
usage(){
echo "
usage: $0 -h | -l | -b <branch> | -k <URL> | -m <email@address.com>
If the -k option is not used the latest will be used
OPTIONS:
-h Shows this message
-l List Branches
-b Branch
-k Koji kernel url to download
-m address to mail
EXAMPLE:
$0 -h
$0 -l
$0 -b 3.11.y
$0 -k http://japan.proximity.on.ca/kojifiles/packages/raspberrypi...
$0 -b 3.12.y -k http://japan.proximity.on.ca/...
$0 -m somebody@senecacollege.ca
"
}
sendmail(){
echo "There is a new kernel" | mail -s "Kernel Update" $1
}
# Use options to find out which version of the kernel to use
# Also added an option to list all the branches
while getopts "hlb:k:m:" OPTION
do
case $OPTION in
h)
usage
exit 1
;;
b)
branch=$OPTARG
;;
k)
koji_url=$OPTARG
;;
l)
cat ./branches
exit 1
;;
m)
mailFlag=1
email_address=$OPTARG
;;
# v)
# versionFlag=1
# version=$OPTARG
# ;;
?)
usage
exit 1
;;
esac
done
# Find version from koji url, if the -k option is not in use
if [ "$versionFlag" != "1" ]; then
version=$( echo $koji_url | cut -d '/' -f7 | cut -d '-' -f2)
fi
# Find source rpm file name downloaded from koji
srcrpm=$( echo $koji_url | cut -d '/' -f10 )
distro=$( echo $srcrpm | cut -d '.' -f5)
# If the file is empty store the long_commit variable
# Otherwise compare the long commit form Github with the one in the file
# If they are the same do nothing and exit
# If they are different store the new value in the file and download the new kernel
if [ ! -s ./long_commit ]; then
echo $long_commit > ./long_commit
else
if [ "$long_commit" == "$stored_long_commit" ] ; then
#Nothing to do
flag=0
echo "There is no new kernel"
else
# Replace the old commit with new commit
echo $long_commit > ./long_commit
#Proceed to build kernel
flag=1
fi
fi
if [ $flag -eq 1 ]; then
# Download previous built kernel to retrieve some necessary files including the spec file
wget $koji_url
# Download new kernel
new_kernel_url=$(echo "https://github.com/raspberrypi/linux/tarball/$long_commit")
wget $new_kernel_url
# Create and wipe rmbuild tree of directories
rpmdev-setuptree
rpmdev-wipetree
# Extract files from source rpm downloaded from koji
rpm2cpio $srcrpm | cpio -idmv
# Modifiying SPEC file with info from the new kernel
sed -i "s/commit_date\ .*/commit_date $commit_date/" raspberrypi-kernel.spec
sed -i "s/commit_short\ .*/commit_short $short_commit/" raspberrypi-kernel.spec
sed -i "s/commit_long\ .*/commit_long $long_commit/" raspberrypi-kernel.spec
# Copy spec file to ~/rpmbuild/SPECS
mv raspberrypi-kernel.spec $topdir/SPECS/
# Bump SPEC
rpmdev-bumpspec -c "Updated to latest commit" $topdir/SPECS/raspberrypi-kernel.spec
# Copye source files to ~/rpmbuild/SOURCES
mv first32k.bin.bz2 $topdir/SOURCES
newfile=$(echo "pidora-config-$version-"$short_commit".bz2")
file=$( ls -l | grep 'pidora-config' | awk '{print $9}' )
mv $file $newfile
mv $newfile $topdir/SOURCES
mv "$long_commit" $topdir/SOURCES
# Send Source Package to Koji
release=$( cat ~/rpmbuild/SPECS/raspberrypi-kernel.spec | grep Release | cut -d ' ' -f9 | cut -d '.' -f1)
modular_srcrpm=$(echo "raspberrypi-kernel-"$version"-"$release"."$commit_date"git"$short_commit"."$distro".src.rpm")
koji -s 'http://japan.proximity.on.ca/kojihub' build f18-rpfr-updates-automated $topdir/SRPMS/$modular_srcrpm
# Send email reporting about new build
if [ "$mailFlag" == "1" ]; then
sendmail $email_address
fi
fi
Wednesday, November 27, 2013
Release 0.2
In this release I added Options which will provide help with the -h option, -l will list the branches, -b will specify the branch and -k will specify which url to use to download the koji source rpm.
If no option is selected the script will use the latest branch and it will select a pre-configured url to get the koji source rpm.
#!/bin/bash
#Check if the file contain the commit long exists
#If it does not exist create the file
if [ ! -f ./branches ]; then
touch branches
fi
if [ ! -f ./long_commit ]; then
touch long_commit
fi
# Retrieve branches from github - Retrieving branches from ls-remote instead of wget
#wget -qO- https://github.com/raspberrypi/linux | cat | grep 'data-name="rpi' | cut -d '"' -f2 > ./branches
# Delete the patches branch - Used only when using the wget way
#sed -i '$ d' ./branches
# Storing git remote head values on a file
git ls-remote https://github.com/raspberrypi/linux.git > ls_remote
# Retrieving Branches from ls_remote
# The sort is only cheking for the decimal place if a version 4.x comes out this will stop working if 3.x versions
# remain when doing a git ls-remote
cat ls_remote | grep rpi | cut -d '/' -f3 | sort -t $'.' -n -k2,2 > ./branches
# Store latest branch in a variable
branch=$(tail -1 ./branches)
# Storing url for use with wget
url=$(echo "https://github.com/raspberrypi/linux/tree/$branch/kernel")
wget -qO- $url > ./page_source
# Retrieving informacion from the git ls-remote command
#commit_date=$( cat ./page_source | grep 'committed <time class="js-relative-date"' | cut -d '"' -f6 | cut -d ' ' -f1 | cut -d '-' -f1-3 --output-delimiter='' )
commit_date=$( date +"%Y%m%d")
long_commit=$( cat ./ls_remote | grep $branch | cut -f1)
short_commit=$( echo $long_commit | cut -c 1-7)
stored_long_commit=$(cat ./long_commit)
# Finding out the %_topdir value to see where the rpmbuild directory is being created
topdir=$( rpm --eval '%{_topdir}')
# Find latest package submitted to rpfr18-updates-automated
koji_build=$( armv6-koji latest-pkg f18-rpfr-updates-automated raspberrypi-kernel | tail -1 | cut -d ' ' -f1)
# Safe to use koji URL
koji_url=$( echo "http://japan.proximity.on.ca/kojifiles/packages/raspberrypi-kernel/3.11.6/4.20131023git10bc582.rpfr19/src/raspberrypi-kernel-3.11.6-4.20131023git10bc582.rpfr19.src.rpm" )
# Usage function for the script
usage(){
echo "
usage: $0 -h | -l | -b <branch> | -k <URL>
If the -k option is not used the latest will be used
OPTIONS:
-h Shows this message
-l List Branches
-b Branch
If no option is selected the script will use the latest branch and it will select a pre-configured url to get the koji source rpm.
#!/bin/bash
#Check if the file contain the commit long exists
#If it does not exist create the file
if [ ! -f ./branches ]; then
touch branches
fi
if [ ! -f ./long_commit ]; then
touch long_commit
fi
# Retrieve branches from github - Retrieving branches from ls-remote instead of wget
#wget -qO- https://github.com/raspberrypi/linux | cat | grep 'data-name="rpi' | cut -d '"' -f2 > ./branches
# Delete the patches branch - Used only when using the wget way
#sed -i '$ d' ./branches
# Storing git remote head values on a file
git ls-remote https://github.com/raspberrypi/linux.git > ls_remote
# Retrieving Branches from ls_remote
# The sort is only cheking for the decimal place if a version 4.x comes out this will stop working if 3.x versions
# remain when doing a git ls-remote
cat ls_remote | grep rpi | cut -d '/' -f3 | sort -t $'.' -n -k2,2 > ./branches
# Store latest branch in a variable
branch=$(tail -1 ./branches)
# Storing url for use with wget
url=$(echo "https://github.com/raspberrypi/linux/tree/$branch/kernel")
wget -qO- $url > ./page_source
# Retrieving informacion from the git ls-remote command
#commit_date=$( cat ./page_source | grep 'committed <time class="js-relative-date"' | cut -d '"' -f6 | cut -d ' ' -f1 | cut -d '-' -f1-3 --output-delimiter='' )
commit_date=$( date +"%Y%m%d")
long_commit=$( cat ./ls_remote | grep $branch | cut -f1)
short_commit=$( echo $long_commit | cut -c 1-7)
stored_long_commit=$(cat ./long_commit)
# Finding out the %_topdir value to see where the rpmbuild directory is being created
topdir=$( rpm --eval '%{_topdir}')
# Find latest package submitted to rpfr18-updates-automated
koji_build=$( armv6-koji latest-pkg f18-rpfr-updates-automated raspberrypi-kernel | tail -1 | cut -d ' ' -f1)
# Safe to use koji URL
koji_url=$( echo "http://japan.proximity.on.ca/kojifiles/packages/raspberrypi-kernel/3.11.6/4.20131023git10bc582.rpfr19/src/raspberrypi-kernel-3.11.6-4.20131023git10bc582.rpfr19.src.rpm" )
# Usage function for the script
usage(){
echo "
usage: $0 -h | -l | -b <branch> | -k <URL>
If the -k option is not used the latest will be used
OPTIONS:
-h Shows this message
-l List Branches
-b Branch
-k Koji kernel url to download
EXAMPLE:
$0 -h
$0 -l
$0 -b 3.11.y
$0 -k http://japan.proximity.on.ca/kojifiles/packages/raspberrypi...
$0 -b 3.12.y -k http://japan.proximity.on.ca/...
"
}
# Use options to find out which version of the kernel to use
# Also added an option to list all the branches
while getopts "hlb:k:" OPTION
do
case $OPTION in
h)
usage
exit 1
;;
b)
branch=$OPTARG
;;
k)
koji_url=$OPTARG
;;
l)
cat ./branches
exit 1
;;
?)
usage
exit 1
;;
esac
done
# If the file is empty store the long_commit variable
# Otherwise compare the long commit form Github with the one in the file
# If they are the same do nothing and exit
# If they are different store the new value in the file and download the new kernel
if [ ! -s ./long_commit ]; then
echo $long_commit > ./long_commit
else
if [ "$long_commit" == "$stored_long_commit" ] ; then
#Nothing to do
flag=0
echo "There is no new kernel"
else
# Replace the old commit with new commit
echo $long_commit > ./long_commit
#Proceed to build kernel
flag=1
fi
fi
if [ $flag -eq 1 ]; then
# Download previous built kernel to retrieve some necessary files including the spec file
wget $koji_url
# Download new kernel
new_kernel_url=$(echo "https://github.com/raspberrypi/linux/tarball/$long_commit")
wget $new_kernel_url
# Create and wipe rmbuild tree of directories
rpmdev-setuptree
rpmdev-wipetree
# Extract files from source rpm downloaded from koji
rpm2cpio raspberrypi-kernel-3.11.6-4.20131023git10bc582.rpfr19.src.rpm | cpio -idmv
# Modifiying SPEC file with info from the new kernel
sed -i "s/commit_date\ .*/commit_date $commit_date/" raspberrypi-kernel.spec
sed -i "s/commit_short\ .*/commit_short $short_commit/" raspberrypi-kernel.spec
sed -i "s/commit_long\ .*/commit_long $long_commit/" raspberrypi-kernel.spec
# Copy spec file to ~/rpmbuild/SPECS
mv raspberrypi-kernel.spec $topdir/SPECS/
# Bump SPEC
rpmdev-bumpspec -c "Updated to latest commit" $topdir/SPECS/raspberrypi-kernel.spec
# Copye source files to ~/rpmbuild/SOURCES
mv first32k.bin.bz2 $topdir/SOURCES
newfile=$(echo "pidora-config-3.11.6-"$short_commit".bz2")
file=$( ls -l | grep 'pidora-config' | awk '{print $9}' )
mv $file $newfile
mv $newfile $topdir/SOURCES
mv "$long_commit" $topdir/SOURCES
# Build rpm source package
rpmbuild -bs $topdir/SPECS/raspberrypi-kernel.spec
# Send Source Package to Koji
release=$( cat ~/rpmbuild/SPECS/raspberrypi-kernel.spec | grep Release | cut -d ' ' -f9 | cut -d '.' -f1)
srcrpm=$(echo "raspberrypi-kernel-3.11.6-"$release"."$commit_date"git"$short_commit".rpfr19.src.rpm")
koji -s 'http://japan.proximity.on.ca/kojihub' build f18-rpfr-updates-automated $topdir/SRPMS/$srcrpm
fi
Tuesday, November 26, 2013
Release 0.1.5
- After seeing that wget was not actually getting the source how it was supposed to I made a few changes. So instead of using wget to retrieve commits and the kernel versions I am using git ls-remote it is way better.
- Date now is using the build date instead of the date of the commit
- Instead of looking for the rpmbuild folder in home which is the default directory I am retrieving the information from the %{_topdir} variable. This way if somebody has setup the rmpbuild directory to another location the script will still be able to function properly.
--------------------------------------------------
Start of script
#!/bin/bash
#Check if the file contain the commit long exists
#If it does not exist create the file
if [ ! -f ./branches ]; then
touch branches
fi
if [ ! -f ./long_commit ]; then
touch long_commit
fi
if [ ! -f ./page_source ]; then
touch page_source
fi
# Retrieve branches from github - Retrieving branches from ls-remote instead of wget
#wget -qO- https://github.com/raspberrypi/linux | cat | grep 'data-name="rpi' | cut -d '"' -f2 > ./branches
# Delete the patches branch - Used only when using the wget way
#sed -i '$ d' ./branches
# Storing git remote head values on a file
git ls-remote https://github.com/raspberrypi/linux.git > ls_remote
# Retrieving Branches from ls_remote
# The sort is only cheking for the decimal place if a version 4.x comes out this will stop working if 3.x versions
# remain when doing a git ls-remote
cat ls_remote | grep rpi | cut -d '/' -f3 | sort -t $'.' -n -k2,2 > ./branches
# Store latest branch in a variable
branch=$(tail -1 ./branches)
# There are some issues with 3.12.y so I am statically using 3.11.y
branch="rpi-3.11.y"
# Storing url for use with wget
url=$(echo "https://github.com/raspberrypi/linux/tree/$branch/kernel")
wget -qO- $url > ./page_source
# Retrieving informacion from the page source
#commit_date=$( cat ./page_source | grep 'committed <time class="js-relative-date"' | cut -d '"' -f6 | cut -d ' ' -f1 | cut -d '-' -f1-3 --output-delimiter='' )
commit_date=$( date +"%Y%m%d")
long_commit=$( cat ./ls_remote | grep $branch | cut -f1)
short_commit=$( echo $long_commit | cut -c 1-7)
stored_long_commit=$(cat ./long_commit)
# Finding out the %_topdir value to see where the rpmbuild directory is being created
topdir=$( rpm --eval '%{_topdir}')
# If the file is empty store the long_commit variable
# Otherwise compare the long commit form Github with the one in the file
# If they are the same do nothing and exit
# If they are different store the new value in the file and download the new kernel
if [ ! -s ./long_commit ]; then
echo $long_commit > ./long_commit
else
if [ "$long_commit" == "$stored_long_commit" ] ; then
#Nothing to do
flag=0
echo "There is no new kernel"
else
# Replace the old commit with new commit
echo $long_commit > ./long_commit
#Proceed to build kernel
flag=1
fi
fi
if [ $flag -eq 1 ]; then
# Download previous built kernel to retrieve some necessary files including the spec file
wget http://japan.proximity.on.ca/kojifiles/packages/raspberrypi-kernel/3.11.6/4.20131023git10bc582.rpfr19/src/raspberrypi-kernel-3.11.6-4.20131023git10bc582.rpfr19.src.rpm
# Download new kernel
new_kernel_url=$(echo "https://github.com/raspberrypi/linux/tarball/$long_commit")
wget $new_kernel_url
rpmdev-setuptree
rpmdev-wipetree
rpm2cpio raspberrypi-kernel-3.11.6-4.20131023git10bc582.rpfr19.src.rpm | cpio -idmv
# rpm -i raspberrypi-kernel-3.11.6-4.20131023git10bc582.rpfr19.src.rpm
# Modifiying SPEC file with info from the new kernel
sed -i "s/commit_date\ .*/commit_date $commit_date/" raspberrypi-kernel.spec
sed -i "s/commit_short\ .*/commit_short $short_commit/" raspberrypi-kernel.spec
sed -i "s/commit_long\ .*/commit_long $long_commit/" raspberrypi-kernel.spec
# Copy spec file to ~/rpmbuild/SPECS
cp raspberrypi-kernel.spec $topdir/SPECS/
# Bump SPEC
rpmdev-bumpspec -c "Updated to latest commit" $topdir/SPECS/raspberrypi-kernel.spec
# Copye source files to ~/rpmbuild/SOURCES
cp first32k.bin.bz2 $topdir/SOURCES
newfile=$(echo "pidora-config-3.11.6-"$short_commit".bz2")
file=$( ls -l | grep 'pidora-config' | awk '{print $9}' )
cp $file $newfile
mv $newfile $topdir/SOURCES
cp "$long_commit" $topdir/SOURCES
# Build rpm source package
rpmbuild -bs $topdir/SPECS/raspberrypi-kernel.spec
# Send Source Package to Koji
release=$( cat ~/rpmbuild/SPECS/raspberrypi-kernel.spec | grep Release | cut -d ' ' -f9 | cut -d '.' -f1)
srcrpm=$(echo "raspberrypi-kernel-3.11.6-"$release"."$commit_date"git"$short_commit".rpfr19.src.rpm")
# koji -s 'http://japan.proximity.on.ca/kojihub' build f18-rpfr-updates-automated $topdir/SRPMS/$srcrpm
fi
Wednesday, November 20, 2013
Release 0.1.1
I have modified my release 0.1 to send to koji the right source package, before it was looking for the wrong one. I could not test this because I had not setup the keys to access the koji system in seneca.
The lines modified are:
# Send Source Package to Koji
release=$( cat ~/rpmbuild/SPECS/raspberrypi-kernel.spec | grep Release | cut -d ' ' -f9 | cut -d '.' -f1)
srcrpm=$(echo "raspberrypi-kernel-3.11.6-"$release"."$commit_date"git"$short_commit".rpfr19.src.rpm")
koji -s 'http://japan.proximity.on.ca/kojihub' build f18-rpfr-updates-automated ~/rpmbuild/SRPMS/$srcrpm
The lines modified are:
# Send Source Package to Koji
release=$( cat ~/rpmbuild/SPECS/raspberrypi-kernel.spec | grep Release | cut -d ' ' -f9 | cut -d '.' -f1)
srcrpm=$(echo "raspberrypi-kernel-3.11.6-"$release"."$commit_date"git"$short_commit".rpfr19.src.rpm")
koji -s 'http://japan.proximity.on.ca/kojihub' build f18-rpfr-updates-automated ~/rpmbuild/SRPMS/$srcrpm
When sent to koji I get the following error
ActionNotAllowed: policy violation (build_from_srpm)
Perhaps it is because I am not yet a user in the koji system or maybe because I am taking the file from SRPMS.
I have also started looking into getops for 0.2 and a few other tweaks for gathering the information and files needed.
Wednesday, November 13, 2013
Release 0.1
After doing some spending some time with my code given that the presentation day was upon me, I managed to get the most of the code up and running with the exception of the last command koji that was not accepting my keys.
There are other things to tinker in the code but for 0.1 this should suffice.
#!/bin/bash
#Check if the file contain the commit long exists
#If it does not exist create the file
if [ ! -f ./branches ]; then
touch branches
fi
if [ ! -f ./long_commit ]; then
touch long_commit
fi
if [ ! -f ./page_source ]; then
touch page_source
fi
# Retrieve branches from github
wget -qO- https://github.com/raspberrypi/linux | cat | grep 'data-name="rpi' | cut -d '"' -f2 > ./branches
# Delete the patches branci that way the latest branch is last
sed -i '$ d' ./branches
# store in a variable the latest branch
branch=$(tail -1 ./branches)
# There are some issues with 3.12.y so I am statically using 3.11.y
branch="rpi-3.11.y"
# Storing url for use with wget
url=$(echo "https://github.com/raspberrypi/linux/tree/$branch/kernel")
wget -qO- $url > ./page_source
commit_date=$( cat ./page_source | grep 'committed <time class="js-relative-date"' | cut -d '"' -f6 | cut -d ' ' -f1 | cut -d '-' -f1-3 --output-delimiter='' )
long_commit=$( cat ./page_source | grep 'js-zeroclipboard zeroclipboard-link' | cut -d '"' -f4)
short_commit=$( echo $long_commit | cut -c 1-7)
stored_long_commit=$(cat ./long_commit)
# If the file is empty store the long_commit variable
# Otherwise compare the long commit form Github with the one in the file
# If they are the same do nothing and exit
# If they are different store the new value in the file and download the new kernel
if [ ! -s ./long_commit ]; then
echo $long_commit > ./long_commit
else
if [ "$long_commit" == "$stored_long_commit" ] ; then
#Nothing to do
flag=0
echo "There is no new kernel"
else
# Replace the old commit with new commit
echo $long_commit > ./long_commit
#Proceed to build kernel
flag=1
fi
fi
if [ $flag -eq 1 ]; then
There are other things to tinker in the code but for 0.1 this should suffice.
#!/bin/bash
#Check if the file contain the commit long exists
#If it does not exist create the file
if [ ! -f ./branches ]; then
touch branches
fi
if [ ! -f ./long_commit ]; then
touch long_commit
fi
if [ ! -f ./page_source ]; then
touch page_source
fi
# Retrieve branches from github
wget -qO- https://github.com/raspberrypi/linux | cat | grep 'data-name="rpi' | cut -d '"' -f2 > ./branches
# Delete the patches branci that way the latest branch is last
sed -i '$ d' ./branches
# store in a variable the latest branch
branch=$(tail -1 ./branches)
# There are some issues with 3.12.y so I am statically using 3.11.y
branch="rpi-3.11.y"
# Storing url for use with wget
url=$(echo "https://github.com/raspberrypi/linux/tree/$branch/kernel")
wget -qO- $url > ./page_source
commit_date=$( cat ./page_source | grep 'committed <time class="js-relative-date"' | cut -d '"' -f6 | cut -d ' ' -f1 | cut -d '-' -f1-3 --output-delimiter='' )
long_commit=$( cat ./page_source | grep 'js-zeroclipboard zeroclipboard-link' | cut -d '"' -f4)
short_commit=$( echo $long_commit | cut -c 1-7)
stored_long_commit=$(cat ./long_commit)
# If the file is empty store the long_commit variable
# Otherwise compare the long commit form Github with the one in the file
# If they are the same do nothing and exit
# If they are different store the new value in the file and download the new kernel
if [ ! -s ./long_commit ]; then
echo $long_commit > ./long_commit
else
if [ "$long_commit" == "$stored_long_commit" ] ; then
#Nothing to do
flag=0
echo "There is no new kernel"
else
# Replace the old commit with new commit
echo $long_commit > ./long_commit
#Proceed to build kernel
flag=1
fi
fi
if [ $flag -eq 1 ]; then
# Download previous built kernel to retrieve some necessary files including the spec file
wget http://japan.proximity.on.ca/kojifiles/packages/raspberrypi-kernel/3.11.6/4.20131023git10bc582.rpfr19/src/raspberrypi-kernel-3.11.6-4.20131023git10bc582.rpfr19.src.rpm
# Download new kernel
new_kernel_url=$(echo "https://github.com/raspberrypi/linux/tarball/$long_commit")
wget $new_kernel_url
rpmdev-setuptree
rpmdev-wipetree
rpm2cpio raspberrypi-kernel-3.11.6-4.20131023git10bc582.rpfr19.src.rpm | cpio -idmv
# rpm -i raspberrypi-kernel-3.11.6-4.20131023git10bc582.rpfr19.src.rpm
# Modifiying SPEC file with info from the new kernel
sed -i "s/commit_date\ .*/commit_date $commit_date/" raspberrypi-kernel.spec
sed -i "s/commit_short\ .*/commit_short $short_commit/" raspberrypi-kernel.spec
sed -i "s/commit_long\ .*/commit_long $long_commit/" raspberrypi-kernel.spec
# Copy spec file to ~/rpmbuild/SPECS
cp raspberrypi-kernel.spec ~/rpmbuild/SPECS/
# Bump SPEC
rpmdev-bumpspec -c "Updated to latest commit" ~/rpmbuild/SPECS/raspberrypi-kernel.spec
# Copye source files to ~/rpmbuild/SOURCES
cp first32k.bin.bz2 ~/rpmbuild/SOURCES
newfile=$(echo "pidora-config-3.11.6-"$short_commit".bz2")
file=$( ls -l | grep 'pidora-config' | awk '{print $9}' )
cp $file $newfile
mv $newfile ~/rpmbuild/SOURCES
cp "$long_commit" ~/rpmbuild/SOURCES
# Build rpm source package
rpmbuild -bs ~/rpmbuild/SPECS/raspberrypi-kernel.spec
# Send Source Package to Koji
koji -s 'http://japan.proximity.on.ca/kojihub' build f18-rpfr-updates-automated-build ~/rpmbuild/SRPMS/raspberrypi-kernel-3.11.6-5.20131013git3e1fee6.rpfr19.src.rpm
fi
There it is my code. I got to say, I was a rusty with bash
Thursday, October 3, 2013
Python Lab
Python lab:
Python from the little I have played with python it shares some similarities with java. I do appreciate the fact that it forces you indent your code and make it closer to be universal but I do miss starting and ending brackets. I also miss ending code statements with ; This means that I cannot put another line of code in the same line. Variables are also handled differently as you do not need to setup the type for them. This could maybe become a little bit confusing if not being careful because the variable could switch from int to string for example well if python permits that.
In general it is easy to start coding with python if there is a previous java/c experience. For the part Google is kind enough to help.
17 import random
18 import os
19 import sys
20
21 #Intro and tutorial
22 os.system('clear')
23 print ("#####################")
24 print ("#Rules of the game: #")
25 print ("#####################\n")
26 print ("Guess the right number from 1 to 100")
27 print ("The maximum number of tries is 15, good luck !!!\n")
28
29 try:
30 raw_input("press enter to continue...")
31 # Handle interrupts like ctrl+c
32 except KeyboardInterrupt:
33 print ("\nUntil next time quitter")
34 sys.exit()
35
36 # Initialize vars
37 guess = 0
38 numTries = 0
39 secret=random.randint(1,100)
40
41 # Loop until answer is found or number of tries gets to 15
42 while (secret != guess and numTries != 15):
43 # Catch exceptions
44 try:
45 guess = int(raw_input("\nEnter a guess: "))
46 if guess < secret:
47 numTries = numTries + 1
48 print "Too low! , Try #:", numTries
49 elif guess > secret:
50 numTries = numTries + 1
51 print "Too high! , Try #:", numTries
52 else:
53 numTries = numTries + 1
54 print "Correct!, Total number of tries:", numTries , "\n"
55 if numTries == 15:
56 print "\nMaximum number of tries reached, Good luck next time !!!\n"
57
58 # Handle characters other than int
59 except ValueError:
60 print ("Please enter integers only")
61
62 # Handle interrupts like ctrl+c
63 except KeyboardInterrupt:
64 print ("\nUntil next time quitter")
65 sys.exit()
Here is my code. I did go a little bit further while playing with python. It is was really enjoyable. Of course improvements could be made … perhaps … but that is beyond the scope of this lab.
I went ahead and did everything in one commit. It was short enough to keep track of everything going in the code and find the errors instantly.
rhernandez2@matrix:~/play/guess> git commit -a -m "Initial and maybe last commit"
[master 430ea2a] Initial and maybe last commit
Committer: Ronald Hernandez <rhernandez2@matrix.senecac.on.ca>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:
git config --global user.name "Your Name"
git config --global user.email you@example.com
After doing this, you may fix the identity used for this commit with:
git commit --amend --reset-author
1 file changed, 52 insertions(+), 8 deletions(-)
Thursday, September 26, 2013
Mock Lab
The mock lab was flawless for both of the packages (which and less)
Here is the output of which
INFO: mock.py version 1.1.33 starting...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
INFO: Start(less-458-4.fc19.src.rpm) Config(fedora-19-x86_64)
Start: lock buildroot
Start: clean chroot
Finish: clean chroot
Finish: lock buildroot
Start: chroot init
Start: lock buildroot
Mock Version: 1.1.33
INFO: Mock Version: 1.1.33
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled yum cache
Start: cleaning yum metadata
Finish: cleaning yum metadata
INFO: enabled ccache
Start: device setup
Finish: device setup
Start: yum update
Start: Outputting list of available packages
Finish: Outputting list of available packages
Finish: yum update
Start: creating cache
Finish: creating cache
Finish: lock buildroot
Finish: chroot init
INFO: Installed packages:
Start: build phase for less-458-4.fc19.src.rpm
Start: device setup
Finish: device setup
Start: build setup for less-458-4.fc19.src.rpm
Finish: build setup for less-458-4.fc19.src.rpm
Start: rpmbuild -bb less-458-4.fc19.src.rpm
Start: Outputting list of installed packages
Finish: Outputting list of installed packages
Finish: rpmbuild -bb less-458-4.fc19.src.rpm
Finish: build phase for less-458-4.fc19.src.rpm
INFO: Done(less-458-4.fc19.src.rpm) Config(fedora-19-x86_64) 5 minutes 8 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-19-x86_64/result
Finish: run
[ron@f19 SRPMS]$ cd ~/Downloads/
[ron@f19 Downloads]$ mock -r fedora-19-x86_64 which-2.20-1.fc19.src.rpm
INFO: mock.py version 1.1.33 starting...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
INFO: Start(which-2.20-1.fc19.src.rpm) Config(fedora-19-x86_64)
Start: lock buildroot
Start: clean chroot
INFO: chroot (/var/lib/mock/fedora-19-x86_64) unlocked and deleted
Finish: clean chroot
Finish: lock buildroot
Start: chroot init
Start: lock buildroot
Mock Version: 1.1.33
INFO: Mock Version: 1.1.33
INFO: calling preinit hooks
INFO: enabled root cache
Start: unpacking root cache
Finish: unpacking root cache
INFO: enabled yum cache
Start: cleaning yum metadata
Finish: cleaning yum metadata
INFO: enabled ccache
Start: device setup
Finish: device setup
Start: yum update
Start: Outputting list of available packages
Finish: Outputting list of available packages
Finish: yum update
Finish: lock buildroot
Finish: chroot init
INFO: Installed packages:
Start: build phase for which-2.20-1.fc19.src.rpm
Start: device setup
Finish: device setup
Start: build setup for which-2.20-1.fc19.src.rpm
Finish: build setup for which-2.20-1.fc19.src.rpm
Start: rpmbuild -bb which-2.20-1.fc19.src.rpm
Start: Outputting list of installed packages
Finish: Outputting list of installed packages
Finish: rpmbuild -bb which-2.20-1.fc19.src.rpm
Finish: build phase for which-2.20-1.fc19.src.rpm
INFO: Done(which-2.20-1.fc19.src.rpm) Config(fedora-19-x86_64) 0 minutes 33 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-19-x86_64/result
Finish: run
I was really glad I switched gimp for which … It would have been a terrible pain to build an RPM package of gimp and then run mock with all the dependencies that gimp requires and the dependencies of dependencies.
Mock did not take too long to finish, perhaps a little bit more than building the RPM package. I was expecting mock to finish successfully for both packages since I spent a lot of time looking working on the spec file of each one.
--------------------------------------
Koji Lab
For the koji lab first I yum install fedora-packager then
I cd to /usr/bin and run ./fedora-packager-setup.
It prompts me for my FAS account and then It asks for an export password.
Afterwards I add the certificate to chrome and enter the export password. I test that I can sign in with koji. Everything seems good to go.
Testing less with Koji
I begin with the following command:
time s390-koji build f19 --scratch less-458-4.fc19.src.rpm
this is what I get back:
Created task: 1217365
Task info: http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=1217365
Watching tasks (this may be safely interrupted)...
1217365 build (f19, less-458-4.fc19.src.rpm): free
1217365 build (f19, less-458-4.fc19.src.rpm): free -> open (fedora2.s390.bos.redhat.com)
1217366 buildArch (less-458-4.fc19.src.rpm, s390): free
1217367 buildArch (less-458-4.fc19.src.rpm, s390x): open (fedora3.s390.bos.redhat.com)
1217366 buildArch (less-458-4.fc19.src.rpm, s390): free -> open (fedora1.s390.bos.redhat.com)
1217366 buildArch (less-458-4.fc19.src.rpm, s390): open (fedora1.s390.bos.redhat.com) -> closed
0 free 2 open 1 done 0 failed
1217367 buildArch (less-458-4.fc19.src.rpm, s390x): open (fedora3.s390.bos.redhat.com) -> closed
0 free 1 open 2 done 0 failed
1217365 build (f19, less-458-4.fc19.src.rpm): open (fedora2.s390.bos.redhat.com) -> closed
0 free 0 open 3 done 0 failed
1217365 build (f19, less-458-4.fc19.src.rpm) completed successfully
real 5m14.094s
user 0m2.188s
sys 0m0.287s
I ran the same source file a second time for f18
real 3m45.686s
user 0m1.957s
sys 0m0.256s
Then for ppc-koji in f18
real 3m49.745s
user 0m1.713s
sys 0m0.234s
ppc-koji in f19
0 free 2 open 1 done 0 failed
gaierror: [Errno -2] Name or service not known
real 3m28.543s
user 0m1.694s
sys 0m0.219s
I got an error or warning for PPC in f19 but since it said 0 failed I ignored it, Nor I have any idea why the powerpc could not recognize what was clearly recognized in all of the other platforms.
time koji build f18 --scratch less-458-4.fc19.src.rpm
The first time I ran koji I forgot to time, so this is the second time I ran koji for the same file
real 2m55.563s
user 0m1.427s
sys 0m0.207s
Now for f19
real 3m5.445s
user 0m1.327s
sys 0m0.227s
Testing which with Koji
koji for f19
real 2m48.838s
user 0m1.152s
sys 0m0.197s
koji for f18
real 2m10.631s
user 0m1.358s
sys 0m0.208s
s390-koji for f18
real 2m58.190s
user 0m1.323s
sys 0m0.146s
ppc-koji for f19
real 2m57.790s
user 0m1.738s
sys 0m0.231s
ppc-koji for f18
real 1m30.340s
user 0m0.935s
sys 0m0.127s
In general Koji took longer to build than mock and rpmbuild for the same packets. Of course we have to take into account that there might not be a free machine always which will increase the time. In my case there was always a free machine but the Seneca network caused the time to increase due to either lag or upload speed etc. Ignoring the network the process is almost as fast as from my computer.
Koji in general is very versatile tool to test if your package will build in several architectures. Given that most likely we have at home a 32 or 64 bit machine. Process is fairly simple and the build time is fast for the packages done.
Subscribe to:
Posts (Atom)