Getting the weekend weather forecast

The last weather forecast option that we are going to implement in our application is the option to get the weather forecast for the upcoming weekend. This implementation is a bit different from the others because the data returned by the weekend's weather is slightly different from today's, five, and ten days weather forecast.

The DOM structure is different and some CSS class names are different as well. If you remember the previous methods that we implemented, we always use the _parser method, which gives us arguments such as the container DOM and a dictionary with the search criteria. The return value of that method is also a dictionary where the key is the class name of the DOM that we were searching and the value is the text within that DOM element. 

Since the CSS class names of the weekend page are different, we need to implement some code to get that array of results and rename all the keys so the _prepare_data function can use scraped results properly.

With that said, let's go ahead and create a new file in the weatherterm/core directory called mapper.py with the following contents:

class Mapper:

    def __init__(self):
        self._mapping = {}

    def _add(self, source, dest):
        self._mapping[source] = dest

    def remap_key(self, source, dest):
        self._add(source, dest)

    def remap(self, itemslist):
        return [self._exec(item) for item in itemslist]

    def _exec(self, src_dict):
        dest = dict()

        if not src_dict:
            raise AttributeError('The source dictionary cannot be  
            empty or None')

        for key, value in src_dict.items():
            try:
                new_key = self._mapping[key]
                dest[new_key] = value
            except KeyError:
                dest[key] = value
        return dest

The Mapper class gets a list with dictionaries and renames specific keys that we would like to rename. The important methods here are remap_key and remap. The remap_key gets two arguments, source and dest. source is the key that we wish to rename and dest is the new name for that key. The remap_key method will add it to an internal dictionary called _mapping, which will be used later on to look up the new key name.

The remap method simply gets a list containing the dictionaries and, for every item on that list, it calls the _exec method that first creates a brand new dictionary, then checks whether the dictionary is empty. In that case, it raises an AttributeError.

If the dictionary has keys, we loop through its items, search for whether the current item's key has a new name in the mapping dictionary. If the new key name is found, will to create a new item with the new key name; otherwise, we just keep the old name. After the loop, the list is returned with all the dictionaries containing the keys with a new name.

Now, we just need to add it to the __init__.py file in the weatherterm/core directory:

from .mapper import Mapper

And, in the weather_com_parser.py file in weatherterm/parsers, we need to import the Mapper:

from weatherterm.core import Mapper

With the mapper in place, we can go ahead and create the _weekend_forecast method in the weather_com_parser.py file, like so:

def _weekend_forecast(self, args):
    criteria = {
        'weather-cell': 'header',
        'temp': 'p',
        'weather-phrase': 'h3',
        'wind-conditions': 'p',
        'humidity': 'p',
    }

    mapper = Mapper()
    mapper.remap_key('wind-conditions', 'wind')
    mapper.remap_key('weather-phrase', 'description')

    content = self._request.fetch_data(args.forecast_option.value,
                                       args.area_code)

    bs = BeautifulSoup(content, 'html.parser')

    forecast_data = bs.find('article', class_='ls-mod')
    container = forecast_data.div.div

    partial_results = self._parse(container, criteria)
    results = mapper.remap(partial_results)

    return self._prepare_data(results, args)

The method starts off by defining the criteria in exactly the same way as the other methods; however, the DOM structure is slightly different and some of the CSS names are also different:

  • weather-cell: Contains the forecast date: FriSEP 29
  • temp: Contains the temperature (high and low): 57°F48°F
  • weather-phrase: Contains the weather conditions: Cloudy
  • wind-conditions: Wind information
  • humidity: The humidity percentage

As you can see, to make it play nicely with the _prepare_data method, we will need to rename some keys in the dictionaries in the result set—wind-conditions should be wind and weather-phrase should be the description.

Luckily, we have introduced the Mapper class to help us out:

mapper = Mapper()
mapper.remap_key('wind-conditions', 'wind')
mapper.remap_key('weather-phrase', 'description')

We create a Mapper object and say, remap wind-conditions to wind and weather-phrase to description:

content = self._request.fetch_data(args.forecast_option.value,
                                   args.area_code)

bs = BeautifulSoup(content, 'html.parser')

forecast_data = bs.find('article', class_='ls-mod')
container = forecast_data.div.div

partial_results = self._parse(container, criteria)

We fetch all the data, create a BeautifulSoup object using the html.parser, and find the container element that contains the children elements that we are interested in. For the weekend forecast, we are interested in getting the article element with a CSS class called ls-mod and within that article we go down to the first child element, which is a DIV, and gets its first child element, which is also a DIV element.

The HTML should look something like this:

<article class='ls-mod'>
  <div>
    <div>
      <!-- this DIV will be our container element -->
    </div>
  </div>
</article>

That's the reason we first find the article, assign it to forecast_data, and then use forecast_data.div.div so we get the DIV element we want.

After defining the container, we pass it to the _parse method together with the container element; when we get the results back, we simply need to run the remap method of the Mapper instance, which will normalize the data for us before we call _prepare_data.

Now, the last detail before we run the application and get the weather forecast for the weekend is that we need to include the --w and --weekend flag to the ArgumentParser. Open the __main__.py file in the weatherterm directory and, just below the --tenday flag, add the following code:

argparser.add_argument('-w', '--weekend',
                       dest='forecast_option',
                       action='store_const',
                       const=ForecastType.WEEKEND,
                       help=('Shows the weather forecast for the 
                             next or '
                             'current weekend'))

Great! Now, run the application using the -w or --weekend flag:

>> [Fri SEP 29]
    High 13.9° / Low 8.9° (Partly Cloudy)
    Wind: ESE 10 mph / Humidity: 79%

>> [Sat SEP 30]
    High 13.9° / Low 9.4° (Partly Cloudy)
    Wind: SE 10 mph / Humidity: 77%

>> [Sun OCT 1]
    High 12.8° / Low 10.6° (Cloudy)
    Wind: SE 14 mph / Humidity: 74%

Note that this time, I used the -u flag to choose Celsius. All the temperatures in the output are represented in Celsius instead of Fahrenheit.

上一章目录下一章