Einstein - Flickr

Get images from Flickr

Write a shell script to get images from the Flickr image sharing site and make a web page to display them.

Background

Flickr is an image sharing site.
(Home page will look broken in lab exam because we block most domains. We unblock just the domains needed to do this practical.)
You can get a feed of images from Flickr.
See Flickr API
For example, latest images tagged with "Louvre" (for the Louvre museum in Paris):
https://api.flickr.com/services/feeds/photos_public.gne?format=json&tags=louvre
Latest images tagged with "Dublin":
https://api.flickr.com/services/feeds/photos_public.gne?format=json&tags=dublin
This data is in JSON format.

The script

Shell script usage ideally would be like:

 flickr (tag)

Get latest images tagged with this tag, and make a web page to display them.
For example:

 flickr belfast
 flickr dublin

Send the output into this file:

 $HOME/flickr.html

Lab exam needs

For reasons explained before, in the lab exam we need extension .sh and call it with "./" in front of it.
Also for this lab we will call it flickr6.

More lab exam needs - Reduce network requests

To reduce network requests, for the lab exam we will not wget the feed each time we run.
Rather we will get a feed once into a file, and then work with that file.
At the end, after we have got the marks, we can put wget back in the script.

Getting started

Right click on one of the feeds above and save into a file called feed.json
The entire practical is now manipulating that file to build a webpage.

Now the structure of your program will be:

# comment and uncomment this line:
exec > [output file]

cat feed.json | [various commands to build a webpage]

And we run it with no argument, just:

 ./flickr6.sh

You can comment and uncomment the "exec" line to switch between output to screen and output to file.

HTML snippets

At first the JSON you get back looks weird.
Then you notice there are snippets of usable HTML in it, like this:
```
 "description": " HTML payload (encoded) ", 
```
For each photo, we are going to extract the descriptions and the list of tags.
You can use egrep to extract lines containing either X or Y using:
```
 egrep 'X|Y' 
```
The patterns we want to look for are:
```
"description":
"tags":
```
Your program will now have a line:
```
cat feed.json | [egrep command]
```
Test that your program now extracts the descriptions and lists of tags, and nothing else.

Fix the snippets

The descriptions are HTML snippets, encoded. Decode them to normal HTML.
You can do this using tr to remove one character. What character?

Your program will now have a line:

cat feed.json | [egrep command] | [tr command]

Set the "exec" command to output to a file. Now double click on the output file and you will actually see the images.

Final tidy

It needs some tidying. Use sed to insert some space before each item:

 
EXTRA="<hr style='margin-top:70'><h1> next photo </h1>"

cat feed.json | [egrep command] | [tr command] | [sed command]

# sed command will change:
#  <p>
# to:
#  $EXTRA<p>
# only make that change once on each line

Now double click on the output file and you will see the images better laid out, with some nice space and a title before each one.

Upload to Einstein

Einstein expects each description line to be all on one line, and each tags line to be all on one line, and there be no other lines:

.. description line ...
.. tags line ...
.. description line ...
.. tags line ...

You can insert extra HTML on a line (like we did with the vertical space). But do not make new lines, or combine lines, with the version you send to Einstein, or Einstein will get confused.

Afterwards (optional)

After you have got your marks, you can fix up the program as follows:

The program will do the wget every time it runs.
Construct the feed URL based on the command-line argument.
You need to surround the URL with quotes because it has character "&" in it.
Remember the two types of quotes.
Use wget to fetch the feed and then go straight to the pipe. No need to have an intermediate file feed.json
You could do some more tidying up. There is some loose text that needs tidying up. You could make lots of improvements.

On a normal setup, it does not need .sh, it can be called anything, and it will be in the PATH.
So we can call it flickr and run it like:

 flickr belfast
 flickr dublin