{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "![](https://raw.githubusercontent.com/rafneta/CienciaDatosPythonCIDE/master/imagenes/banner.png)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Scraping, HTML\n", "\n", "Los componentes de una página web, archivo html, css y javascript (en su forma más sencilla)" ] }, { "cell_type": "raw", "metadata": {}, "source": [ "archivo.html\n", "\n", "\n", "\n", "\n", "\n", "

\n", "Parrafo 1\n", "

\n", "

\n", "Parrafo 2\n", "

\n", "\n", "" ] }, { "cell_type": "raw", "metadata": {}, "source": [ "archivo.css\n", "\n", " .verde{\n", "background:green;\n", "}\n", "\n", ".rojo{\n", "background:red;\n", "}\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import re\n", "import requests\n", "from bs4 import BeautifulSoup" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "b'\\n\\n\\t\\n\\t\\t\\n\\t\\t \\n\\t\\t \\n\\t\\tSe\\xc3\\xb1ales y sistemas: Una perspectiva\\n \\n\\t\\t\\n\\t\\t\\n\\t\\t\"Licencia
Señales y sistemas: una perspectiva por Rafael Martínez-Martínez se distribuye bajo una Licencia Creative Commons Atribución-NoComercial-CompartirIgual 4.0 Internacional.

',\n", " '

Los autores han hecho su mejor esfuerzo en la preparación de este material. Se distribuye gratuitamente con la esperanza de que sea útil, pero sin ninguna garantía expresa o implícita respecto a la exactitud o completitud del contenido.

',\n", " '

Este prefacio (así como todo el documento electrónico) es variante respecto al tiempo, lo que consultes dependerá de la fecha y hora en que lo hagas. Creo (no tengo una referencia que confirme lo que escribiré) que los libros usuales han sido parte de la educación por tradición, es claro que hace décadas (por no decir siglos) una de las formas de almacenar los conocimientos era escribiendo libros y claro que la experiencia de interactuar con un libro es inigualable. En la actualidad tenemos herramientas tecnológicas (tantas que no sabemos que elegir) para almacenar información, transmitir y enseña, simular situaciones, etc. Por tal motivo este \"\"libro\"\" (le llamaremos documento electrónico) es un poco diferente a lo usual. Se presenta de manera escrita la teoría, pero muchas de las explicaciones son mediante video, lo cual acota a que la consulta sea mediante un dispositivo digital (por tal motivo lo definimos como un documento electrónico).

']" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pes1=re.findall(r\"(

[\\s+|\\n+|\\w+]?.+

)\",str(pagina.content,'utf-8'))\n", "print(len(pes1))\n", "pes1[0:3]" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "333\n" ] }, { "data": { "text/plain": [ "[


La licencia es lo siguiente:\n", " \n", " Permite a otros copiar, distribuir, exhibir y ejecutar públicamente el trabajo, así como hacer y distribuir trabajos derivados con fines no comerciales pero reconociendo la autoría y sólo bajo la misma licencia o una compatible.

,\n", "

\"Licencia
Señales y sistemas: una perspectiva por Rafael Martínez-Martínez se distribuye bajo una Licencia Creative Commons Atribución-NoComercial-CompartirIgual 4.0 Internacional.

,\n", "

Los autores han hecho su mejor esfuerzo en la preparación de este material. Se distribuye gratuitamente con la esperanza de que sea útil, pero sin ninguna garantía expresa o implícita respecto a la exactitud o completitud del contenido.

]" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "soup = BeautifulSoup(pagina.content, 'html.parser')\n", "pes=soup.findAll(\"p\")\n", "print(len(pes))\n", "pes[0:3]" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[

\n", " Si \\( \\mathbb{R}e(a)>0\\) la evalución\n", " \\({ \\left. t{ e }^{ -(a+j\\omega )t } \\right| }_{ 0 }^{ \\infty }=0\\),\n", " se tiene que:

,\n", "

\n", " de acuerdo a la definición de valor absoluto\n", "

,\n", "

\n", " Si \\( \\mathbb{R}e(a)>0\\) las evaluciones son \\(1\\) y \\(-1\\)\n", " repectivamente, al operar el signo en el denominador de la \n", " primera expresión se tiene que:

]" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a=soup.findAll(\"p\", class_ = \"justo\")\n", "a" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "scrolled": true, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on Tag in module bs4.element object:\n", "\n", "class Tag(PageElement)\n", " | Tag(parser=None, builder=None, name=None, namespace=None, prefix=None, attrs=None, parent=None, previous=None, is_xml=None, sourceline=None, sourcepos=None, can_be_empty_element=None, cdata_list_attributes=None, preserve_whitespace_tags=None)\n", " | \n", " | Represents an HTML or XML tag that is part of a parse tree, along\n", " | with its attributes and contents.\n", " | \n", " | When Beautiful Soup parses the markup penguin, it will\n", " | create a Tag object representing the tag.\n", " | \n", " | Method resolution order:\n", " | Tag\n", " | PageElement\n", " | builtins.object\n", " | \n", " | Methods defined here:\n", " | \n", " | __bool__(self)\n", " | A tag is non-None even if it has no contents.\n", " | \n", " | __call__(self, *args, **kwargs)\n", " | Calling a Tag like a function is the same as calling its\n", " | find_all() method. Eg. tag('a') returns a list of all the A tags\n", " | found within this tag.\n", " | \n", " | __contains__(self, x)\n", " | \n", " | __copy__(self)\n", " | A copy of a Tag is a new Tag, unconnected to the parse tree.\n", " | Its contents are a copy of the old Tag's contents.\n", " | \n", " | __delitem__(self, key)\n", " | Deleting tag[key] deletes all 'key' attributes for the tag.\n", " | \n", " | __eq__(self, other)\n", " | Returns true iff this Tag has the same name, the same attributes,\n", " | and the same contents (recursively) as `other`.\n", " | \n", " | __getattr__(self, tag)\n", " | Calling tag.subtag is the same as calling tag.find(name=\"subtag\")\n", " | \n", " | __getitem__(self, key)\n", " | tag[key] returns the value of the 'key' attribute for the Tag,\n", " | and throws an exception if it's not there.\n", " | \n", " | __hash__(self)\n", " | Return hash(self).\n", " | \n", " | __init__(self, parser=None, builder=None, name=None, namespace=None, prefix=None, attrs=None, parent=None, previous=None, is_xml=None, sourceline=None, sourcepos=None, can_be_empty_element=None, cdata_list_attributes=None, preserve_whitespace_tags=None)\n", " | Basic constructor.\n", " | \n", " | :param parser: A BeautifulSoup object.\n", " | :param builder: A TreeBuilder.\n", " | :param name: The name of the tag.\n", " | :param namespace: The URI of this Tag's XML namespace, if any.\n", " | :param prefix: The prefix for this Tag's XML namespace, if any.\n", " | :param attrs: A dictionary of this Tag's attribute values.\n", " | :param parent: The PageElement to use as this Tag's parent.\n", " | :param previous: The PageElement that was parsed immediately before\n", " | this tag.\n", " | :param is_xml: If True, this is an XML tag. Otherwise, this is an\n", " | HTML tag.\n", " | :param sourceline: The line number where this tag was found in its\n", " | source document.\n", " | :param sourcepos: The character position within `sourceline` where this\n", " | tag was found.\n", " | :param can_be_empty_element: If True, this tag should be\n", " | represented as . If False, this tag should be represented\n", " | as .\n", " | :param cdata_list_attributes: A list of attributes whose values should\n", " | be treated as CDATA if they ever show up on this tag.\n", " | :param preserve_whitespace_tags: A list of tag names whose contents\n", " | should have their whitespace preserved.\n", " | \n", " | __iter__(self)\n", " | Iterating over a Tag iterates over its contents.\n", " | \n", " | __len__(self)\n", " | The length of a Tag is the length of its list of contents.\n", " | \n", " | __ne__(self, other)\n", " | Returns true iff this Tag is not identical to `other`,\n", " | as defined in __eq__.\n", " | \n", " | __repr__ = __unicode__(self)\n", " | \n", " | __setitem__(self, key, value)\n", " | Setting tag[key] sets the value of the 'key' attribute for the\n", " | tag.\n", " | \n", " | __str__ = __unicode__(self)\n", " | \n", " | __unicode__(self)\n", " | Renders this PageElement as a Unicode string.\n", " | \n", " | childGenerator(self)\n", " | Deprecated generator.\n", " | \n", " | clear(self, decompose=False)\n", " | Wipe out all children of this PageElement by calling extract()\n", " | on them.\n", " | \n", " | :param decompose: If this is True, decompose() (a more\n", " | destructive method) will be called instead of extract().\n", " | \n", " | decode(self, indent_level=None, eventual_encoding='utf-8', formatter='minimal')\n", " | Render a Unicode representation of this PageElement and its\n", " | contents.\n", " | \n", " | :param indent_level: Each line of the rendering will be\n", " | indented this many spaces. Used internally in\n", " | recursive calls while pretty-printing.\n", " | :param eventual_encoding: The tag is destined to be\n", " | encoded into this encoding. This method is _not_\n", " | responsible for performing that encoding. This information\n", " | is passed in so that it can be substituted in if the\n", " | document contains a tag that mentions the document's\n", " | encoding.\n", " | :param formatter: A Formatter object, or a string naming one of\n", " | the standard formatters.\n", " | \n", " | decode_contents(self, indent_level=None, eventual_encoding='utf-8', formatter='minimal')\n", " | Renders the contents of this tag as a Unicode string.\n", " | \n", " | :param indent_level: Each line of the rendering will be\n", " | indented this many spaces. Used internally in\n", " | recursive calls while pretty-printing.\n", " | \n", " | :param eventual_encoding: The tag is destined to be\n", " | encoded into this encoding. decode_contents() is _not_\n", " | responsible for performing that encoding. This information\n", " | is passed in so that it can be substituted in if the\n", " | document contains a tag that mentions the document's\n", " | encoding.\n", " | \n", " | :param formatter: A Formatter object, or a string naming one of\n", " | the standard Formatters.\n", " | \n", " | decompose(self)\n", " | Recursively destroys this PageElement and its children.\n", " | \n", " | This element will be removed from the tree and wiped out; so\n", " | will everything beneath it.\n", " | \n", " | The behavior of a decomposed PageElement is undefined and you\n", " | should never use one for anything, but if you need to _check_\n", " | whether an element has been decomposed, you can use the\n", " | `decomposed` property.\n", " | \n", " | encode(self, encoding='utf-8', indent_level=None, formatter='minimal', errors='xmlcharrefreplace')\n", " | Render a bytestring representation of this PageElement and its\n", " | contents.\n", " | \n", " | :param encoding: The destination encoding.\n", " | :param indent_level: Each line of the rendering will be\n", " | indented this many spaces. Used internally in\n", " | recursive calls while pretty-printing.\n", " | :param formatter: A Formatter object, or a string naming one of\n", " | the standard formatters.\n", " | :param errors: An error handling strategy such as\n", " | 'xmlcharrefreplace'. This value is passed along into\n", " | encode() and its value should be one of the constants\n", " | defined by Python.\n", " | :return: A bytestring.\n", " | \n", " | encode_contents(self, indent_level=None, encoding='utf-8', formatter='minimal')\n", " | Renders the contents of this PageElement as a bytestring.\n", " | \n", " | :param indent_level: Each line of the rendering will be\n", " | indented this many spaces. Used internally in\n", " | recursive calls while pretty-printing.\n", " | \n", " | :param eventual_encoding: The bytestring will be in this encoding.\n", " | \n", " | :param formatter: A Formatter object, or a string naming one of\n", " | the standard Formatters.\n", " | \n", " | :return: A bytestring.\n", " | \n", " | find(self, name=None, attrs={}, recursive=True, text=None, **kwargs)\n", " | Look in the children of this PageElement and find the first\n", " | PageElement that matches the given criteria.\n", " | \n", " | All find_* methods take a common set of arguments. See the online\n", " | documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :param recursive: If this is True, find() will perform a\n", " | recursive search of this PageElement's children. Otherwise,\n", " | only the direct children will be considered.\n", " | :param limit: Stop looking after finding this many results.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | :return: A PageElement.\n", " | :rtype: bs4.element.Tag | bs4.element.NavigableString\n", " | \n", " | findAll = find_all(self, name=None, attrs={}, recursive=True, text=None, limit=None, **kwargs)\n", " | \n", " | findChild = find(self, name=None, attrs={}, recursive=True, text=None, **kwargs)\n", " | \n", " | findChildren = find_all(self, name=None, attrs={}, recursive=True, text=None, limit=None, **kwargs)\n", " | \n", " | find_all(self, name=None, attrs={}, recursive=True, text=None, limit=None, **kwargs)\n", " | Look in the children of this PageElement and find all\n", " | PageElements that match the given criteria.\n", " | \n", " | All find_* methods take a common set of arguments. See the online\n", " | documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :param recursive: If this is True, find_all() will perform a\n", " | recursive search of this PageElement's children. Otherwise,\n", " | only the direct children will be considered.\n", " | :param limit: Stop looking after finding this many results.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | :return: A ResultSet of PageElements.\n", " | :rtype: bs4.element.ResultSet\n", " | \n", " | get(self, key, default=None)\n", " | Returns the value of the 'key' attribute for the tag, or\n", " | the value given for 'default' if it doesn't have that\n", " | attribute.\n", " | \n", " | getText = get_text(self, separator='', strip=False, types=(, ))\n", " | \n", " | get_attribute_list(self, key, default=None)\n", " | The same as get(), but always returns a list.\n", " | \n", " | :param key: The attribute to look for.\n", " | :param default: Use this value if the attribute is not present\n", " | on this PageElement.\n", " | :return: A list of values, probably containing only a single\n", " | value.\n", " | \n", " | get_text(self, separator='', strip=False, types=(, ))\n", " | Get all child strings, concatenated using the given separator.\n", " | \n", " | :param separator: Strings will be concatenated using this separator.\n", " | \n", " | :param strip: If True, strings will be stripped before being\n", " | concatenated.\n", " | \n", " | :types: A tuple of NavigableString subclasses. Any strings of\n", " | a subclass not found in this list will be ignored. By\n", " | default, this means only NavigableString and CData objects\n", " | will be considered. So no comments, processing instructions,\n", " | stylesheets, etc.\n", " | \n", " | :return: A string.\n", " | \n", " | has_attr(self, key)\n", " | Does this PageElement have an attribute with the given name?\n", " | \n", " | has_key(self, key)\n", " | Deprecated method. This was kind of misleading because has_key()\n", " | (attributes) was different from __in__ (contents).\n", " | \n", " | has_key() is gone in Python 3, anyway.\n", " | \n", " | index(self, element)\n", " | Find the index of a child by identity, not value.\n", " | \n", " | Avoids issues with tag.contents.index(element) getting the\n", " | index of equal elements.\n", " | \n", " | :param element: Look for this PageElement in `self.contents`.\n", " | \n", " | prettify(self, encoding=None, formatter='minimal')\n", " | Pretty-print this PageElement as a string.\n", " | \n", " | :param encoding: The eventual encoding of the string. If this is None,\n", " | a Unicode string will be returned.\n", " | :param formatter: A Formatter object, or a string naming one of\n", " | the standard formatters.\n", " | :return: A Unicode string (if encoding==None) or a bytestring \n", " | (otherwise).\n", " | \n", " | recursiveChildGenerator(self)\n", " | Deprecated generator.\n", " | \n", " | renderContents(self, encoding='utf-8', prettyPrint=False, indentLevel=0)\n", " | Deprecated method for BS3 compatibility.\n", " | \n", " | select(self, selector, namespaces=None, limit=None, **kwargs)\n", " | Perform a CSS selection operation on the current element.\n", " | \n", " | This uses the SoupSieve library.\n", " | \n", " | :param selector: A string containing a CSS selector.\n", " | \n", " | :param namespaces: A dictionary mapping namespace prefixes\n", " | used in the CSS selector to namespace URIs. By default,\n", " | Beautiful Soup will use the prefixes it encountered while\n", " | parsing the document.\n", " | \n", " | :param limit: After finding this number of results, stop looking.\n", " | \n", " | :param kwargs: Keyword arguments to be passed into SoupSieve's \n", " | soupsieve.select() method.\n", " | \n", " | :return: A ResultSet of Tags.\n", " | :rtype: bs4.element.ResultSet\n", " | \n", " | select_one(self, selector, namespaces=None, **kwargs)\n", " | Perform a CSS selection operation on the current element.\n", " | \n", " | :param selector: A CSS selector.\n", " | \n", " | :param namespaces: A dictionary mapping namespace prefixes\n", " | used in the CSS selector to namespace URIs. By default,\n", " | Beautiful Soup will use the prefixes it encountered while\n", " | parsing the document.\n", " | \n", " | :param kwargs: Keyword arguments to be passed into SoupSieve's \n", " | soupsieve.select() method.\n", " | \n", " | :return: A Tag.\n", " | :rtype: bs4.element.Tag\n", " | \n", " | smooth(self)\n", " | Smooth out this element's children by consolidating consecutive\n", " | strings.\n", " | \n", " | This makes pretty-printed output look more natural following a\n", " | lot of operations that modified the tree.\n", " | \n", " | ----------------------------------------------------------------------\n", " | Readonly properties defined here:\n", " | \n", " | children\n", " | Iterate over all direct children of this PageElement.\n", " | \n", " | :yield: A sequence of PageElements.\n", " | \n", " | descendants\n", " | Iterate over all children of this PageElement in a\n", " | breadth-first sequence.\n", " | \n", " | :yield: A sequence of PageElements.\n", " | \n", " | isSelfClosing\n", " | Is this tag an empty-element tag? (aka a self-closing tag)\n", " | \n", " | A tag that has contents is never an empty-element tag.\n", " | \n", " | A tag that has no contents may or may not be an empty-element\n", " | tag. It depends on the builder used to create the tag. If the\n", " | builder has a designated list of empty-element tags, then only\n", " | a tag whose name shows up in that list is considered an\n", " | empty-element tag.\n", " | \n", " | If the builder has no designated list of empty-element tags,\n", " | then any tag with no contents is an empty-element tag.\n", " | \n", " | is_empty_element\n", " | Is this tag an empty-element tag? (aka a self-closing tag)\n", " | \n", " | A tag that has contents is never an empty-element tag.\n", " | \n", " | A tag that has no contents may or may not be an empty-element\n", " | tag. It depends on the builder used to create the tag. If the\n", " | builder has a designated list of empty-element tags, then only\n", " | a tag whose name shows up in that list is considered an\n", " | empty-element tag.\n", " | \n", " | If the builder has no designated list of empty-element tags,\n", " | then any tag with no contents is an empty-element tag.\n", " | \n", " | strings\n", " | Yield all strings of certain classes, possibly stripping them.\n", " | \n", " | :param strip: If True, all strings will be stripped before being\n", " | yielded.\n", " | \n", " | :types: A tuple of NavigableString subclasses. Any strings of\n", " | a subclass not found in this list will be ignored. By\n", " | default, this means only NavigableString and CData objects\n", " | will be considered. So no comments, processing instructions,\n", " | etc.\n", " | \n", " | :yield: A sequence of strings.\n", " | \n", " | stripped_strings\n", " | Yield all strings in the document, stripping them first.\n", " | \n", " | :yield: A sequence of stripped strings.\n", " | \n", " | text\n", " | Get all child strings, concatenated using the given separator.\n", " | \n", " | :param separator: Strings will be concatenated using this separator.\n", " | \n", " | :param strip: If True, strings will be stripped before being\n", " | concatenated.\n", " | \n", " | :types: A tuple of NavigableString subclasses. Any strings of\n", " | a subclass not found in this list will be ignored. By\n", " | default, this means only NavigableString and CData objects\n", " | will be considered. So no comments, processing instructions,\n", " | stylesheets, etc.\n", " | \n", " | :return: A string.\n", " | \n", " | ----------------------------------------------------------------------\n", " | Data descriptors defined here:\n", " | \n", " | parserClass\n", " | \n", " | string\n", " | Convenience property to get the single string within this\n", " | PageElement.\n", " | \n", " | TODO It might make sense to have NavigableString.string return\n", " | itself.\n", " | \n", " | :return: If this element has a single string child, return\n", " | value is that string. If this element has one child tag,\n", " | return value is the 'string' attribute of the child tag,\n", " | recursively. If this element is itself a string, has no\n", " | children, or has more than one child, return value is None.\n", " | \n", " | ----------------------------------------------------------------------\n", " | Methods inherited from PageElement:\n", " | \n", " | append(self, tag)\n", " | Appends the given PageElement to the contents of this one.\n", " | \n", " | :param tag: A PageElement.\n", " | \n", " | extend(self, tags)\n", " | Appends the given PageElements to this one's contents.\n", " | \n", " | :param tags: A list of PageElements.\n", " | \n", " | extract(self, _self_index=None)\n", " | Destructively rips this element out of the tree.\n", " | \n", " | :param _self_index: The location of this element in its parent's\n", " | .contents, if known. Passing this in allows for a performance\n", " | optimization.\n", " | \n", " | :return: `self`, no longer part of the tree.\n", " | \n", " | fetchNextSiblings = find_next_siblings(self, name=None, attrs={}, text=None, limit=None, **kwargs)\n", " | \n", " | fetchParents = find_parents(self, name=None, attrs={}, limit=None, **kwargs)\n", " | \n", " | fetchPrevious = find_all_previous(self, name=None, attrs={}, text=None, limit=None, **kwargs)\n", " | \n", " | fetchPreviousSiblings = find_previous_siblings(self, name=None, attrs={}, text=None, limit=None, **kwargs)\n", " | \n", " | findAllNext = find_all_next(self, name=None, attrs={}, text=None, limit=None, **kwargs)\n", " | \n", " | findAllPrevious = find_all_previous(self, name=None, attrs={}, text=None, limit=None, **kwargs)\n", " | \n", " | findNext = find_next(self, name=None, attrs={}, text=None, **kwargs)\n", " | \n", " | findNextSibling = find_next_sibling(self, name=None, attrs={}, text=None, **kwargs)\n", " | \n", " | findNextSiblings = find_next_siblings(self, name=None, attrs={}, text=None, limit=None, **kwargs)\n", " | \n", " | findParent = find_parent(self, name=None, attrs={}, **kwargs)\n", " | \n", " | findParents = find_parents(self, name=None, attrs={}, limit=None, **kwargs)\n", " | \n", " | findPrevious = find_previous(self, name=None, attrs={}, text=None, **kwargs)\n", " | \n", " | findPreviousSibling = find_previous_sibling(self, name=None, attrs={}, text=None, **kwargs)\n", " | \n", " | findPreviousSiblings = find_previous_siblings(self, name=None, attrs={}, text=None, limit=None, **kwargs)\n", " | \n", " | find_all_next(self, name=None, attrs={}, text=None, limit=None, **kwargs)\n", " | Find all PageElements that match the given criteria and appear\n", " | later in the document than this PageElement.\n", " | \n", " | All find_* methods take a common set of arguments. See the online\n", " | documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :param text: A filter for a NavigableString with specific text.\n", " | :param limit: Stop looking after finding this many results.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | :return: A ResultSet containing PageElements.\n", " | \n", " | find_all_previous(self, name=None, attrs={}, text=None, limit=None, **kwargs)\n", " | Look backwards in the document from this PageElement and find all\n", " | PageElements that match the given criteria.\n", " | \n", " | All find_* methods take a common set of arguments. See the online\n", " | documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :param text: A filter for a NavigableString with specific text.\n", " | :param limit: Stop looking after finding this many results.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | :return: A ResultSet of PageElements.\n", " | :rtype: bs4.element.ResultSet\n", " | \n", " | find_next(self, name=None, attrs={}, text=None, **kwargs)\n", " | Find the first PageElement that matches the given criteria and\n", " | appears later in the document than this PageElement.\n", " | \n", " | All find_* methods take a common set of arguments. See the online\n", " | documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :param text: A filter for a NavigableString with specific text.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | :return: A PageElement.\n", " | :rtype: bs4.element.Tag | bs4.element.NavigableString\n", " | \n", " | find_next_sibling(self, name=None, attrs={}, text=None, **kwargs)\n", " | Find the closest sibling to this PageElement that matches the\n", " | given criteria and appears later in the document.\n", " | \n", " | All find_* methods take a common set of arguments. See the\n", " | online documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :param text: A filter for a NavigableString with specific text.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | :return: A PageElement.\n", " | :rtype: bs4.element.Tag | bs4.element.NavigableString\n", " | \n", " | find_next_siblings(self, name=None, attrs={}, text=None, limit=None, **kwargs)\n", " | Find all siblings of this PageElement that match the given criteria\n", " | and appear later in the document.\n", " | \n", " | All find_* methods take a common set of arguments. See the online\n", " | documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :param text: A filter for a NavigableString with specific text.\n", " | :param limit: Stop looking after finding this many results.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | :return: A ResultSet of PageElements.\n", " | :rtype: bs4.element.ResultSet\n", " | \n", " | find_parent(self, name=None, attrs={}, **kwargs)\n", " | Find the closest parent of this PageElement that matches the given\n", " | criteria.\n", " | \n", " | All find_* methods take a common set of arguments. See the online\n", " | documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | \n", " | :return: A PageElement.\n", " | :rtype: bs4.element.Tag | bs4.element.NavigableString\n", " | \n", " | find_parents(self, name=None, attrs={}, limit=None, **kwargs)\n", " | Find all parents of this PageElement that match the given criteria.\n", " | \n", " | All find_* methods take a common set of arguments. See the online\n", " | documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :param limit: Stop looking after finding this many results.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | \n", " | :return: A PageElement.\n", " | :rtype: bs4.element.Tag | bs4.element.NavigableString\n", " | \n", " | find_previous(self, name=None, attrs={}, text=None, **kwargs)\n", " | Look backwards in the document from this PageElement and find the\n", " | first PageElement that matches the given criteria.\n", " | \n", " | All find_* methods take a common set of arguments. See the online\n", " | documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :param text: A filter for a NavigableString with specific text.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | :return: A PageElement.\n", " | :rtype: bs4.element.Tag | bs4.element.NavigableString\n", " | \n", " | find_previous_sibling(self, name=None, attrs={}, text=None, **kwargs)\n", " | Returns the closest sibling to this PageElement that matches the\n", " | given criteria and appears earlier in the document.\n", " | \n", " | All find_* methods take a common set of arguments. See the online\n", " | documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :param text: A filter for a NavigableString with specific text.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | :return: A PageElement.\n", " | :rtype: bs4.element.Tag | bs4.element.NavigableString\n", " | \n", " | find_previous_siblings(self, name=None, attrs={}, text=None, limit=None, **kwargs)\n", " | Returns all siblings to this PageElement that match the\n", " | given criteria and appear earlier in the document.\n", " | \n", " | All find_* methods take a common set of arguments. See the online\n", " | documentation for detailed explanations.\n", " | \n", " | :param name: A filter on tag name.\n", " | :param attrs: A dictionary of filters on attribute values.\n", " | :param text: A filter for a NavigableString with specific text.\n", " | :param limit: Stop looking after finding this many results.\n", " | :kwargs: A dictionary of filters on attribute values.\n", " | :return: A ResultSet of PageElements.\n", " | :rtype: bs4.element.ResultSet\n", " | \n", " | format_string(self, s, formatter)\n", " | Format the given string using the given formatter.\n", " | \n", " | :param s: A string.\n", " | :param formatter: A Formatter object, or a string naming one of the standard formatters.\n", " | \n", " | formatter_for_name(self, formatter)\n", " | Look up or create a Formatter for the given identifier,\n", " | if necessary.\n", " | \n", " | :param formatter: Can be a Formatter object (used as-is), a\n", " | function (used as the entity substitution hook for an\n", " | XMLFormatter or HTMLFormatter), or a string (used to look\n", " | up an XMLFormatter or HTMLFormatter in the appropriate\n", " | registry.\n", " | \n", " | insert(self, position, new_child)\n", " | Insert a new PageElement in the list of this PageElement's children.\n", " | \n", " | This works the same way as `list.insert`.\n", " | \n", " | :param position: The numeric position that should be occupied\n", " | in `self.children` by the new PageElement. \n", " | :param new_child: A PageElement.\n", " | \n", " | insert_after(self, *args)\n", " | Makes the given element(s) the immediate successor of this one.\n", " | \n", " | The elements will have the same parent, and the given elements\n", " | will be immediately after this one.\n", " | \n", " | :param args: One or more PageElements.\n", " | \n", " | insert_before(self, *args)\n", " | Makes the given element(s) the immediate predecessor of this one.\n", " | \n", " | All the elements will have the same parent, and the given elements\n", " | will be immediately before this one.\n", " | \n", " | :param args: One or more PageElements.\n", " | \n", " | nextGenerator(self)\n", " | # Old non-property versions of the generators, for backwards\n", " | # compatibility with BS3.\n", " | \n", " | nextSiblingGenerator(self)\n", " | \n", " | parentGenerator(self)\n", " | \n", " | previousGenerator(self)\n", " | \n", " | previousSiblingGenerator(self)\n", " | \n", " | replaceWith = replace_with(self, replace_with)\n", " | \n", " | replaceWithChildren = unwrap(self)\n", " | \n", " | replace_with(self, replace_with)\n", " | Replace this PageElement with another one, keeping the rest of the\n", " | tree the same.\n", " | \n", " | :param replace_with: A PageElement.\n", " | :return: `self`, no longer part of the tree.\n", " | \n", " | replace_with_children = unwrap(self)\n", " | \n", " | setup(self, parent=None, previous_element=None, next_element=None, previous_sibling=None, next_sibling=None)\n", " | Sets up the initial relations between this element and\n", " | other elements.\n", " | \n", " | :param parent: The parent of this element.\n", " | \n", " | :param previous_element: The element parsed immediately before\n", " | this one.\n", " | \n", " | :param next_element: The element parsed immediately before\n", " | this one.\n", " | \n", " | :param previous_sibling: The most recently encountered element\n", " | on the same level of the parse tree as this one.\n", " | \n", " | :param previous_sibling: The next element to be encountered\n", " | on the same level of the parse tree as this one.\n", " | \n", " | unwrap(self)\n", " | Replace this PageElement with its contents.\n", " | \n", " | :return: `self`, no longer part of the tree.\n", " | \n", " | wrap(self, wrap_inside)\n", " | Wrap this PageElement inside another one.\n", " | \n", " | :param wrap_inside: A PageElement.\n", " | :return: `wrap_inside`, occupying the position in the tree that used\n", " | to be occupied by `self`, and with `self` inside it.\n", " | \n", " | ----------------------------------------------------------------------\n", " | Readonly properties inherited from PageElement:\n", " | \n", " | decomposed\n", " | Check whether a PageElement has been decomposed.\n", " | \n", " | :rtype: bool\n", " | \n", " | next\n", " | The PageElement, if any, that was parsed just after this one.\n", " | \n", " | :return: A PageElement.\n", " | :rtype: bs4.element.Tag | bs4.element.NavigableString\n", " | \n", " | next_elements\n", " | All PageElements that were parsed after this one.\n", " | \n", " | :yield: A sequence of PageElements.\n", " | \n", " | next_siblings\n", " | All PageElements that are siblings of this one but were parsed\n", " | later.\n", " | \n", " | :yield: A sequence of PageElements.\n", " | \n", " | parents\n", " | All PageElements that are parents of this PageElement.\n", " | \n", " | :yield: A sequence of PageElements.\n", " | \n", " | previous\n", " | The PageElement, if any, that was parsed just before this one.\n", " | \n", " | :return: A PageElement.\n", " | :rtype: bs4.element.Tag | bs4.element.NavigableString\n", " | \n", " | previous_elements\n", " | All PageElements that were parsed before this one.\n", " | \n", " | :yield: A sequence of PageElements.\n", " | \n", " | previous_siblings\n", " | All PageElements that are siblings of this one but were parsed\n", " | earlier.\n", " | \n", " | :yield: A sequence of PageElements.\n", " | \n", " | ----------------------------------------------------------------------\n", " | Data descriptors inherited from PageElement:\n", " | \n", " | __dict__\n", " | dictionary for instance variables (if defined)\n", " | \n", " | __weakref__\n", " | list of weak references to the object (if defined)\n", " | \n", " | nextSibling\n", " | \n", " | previousSibling\n", "\n" ] } ], "source": [ "help(a[1])" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'\\n de acuerdo a la definición de valor absoluto\\n '" ] }, "execution_count": 51, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a[1].text" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'de acuerdo a la definición de valor absoluto'" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "texto=re.findall(r\"[^\\n].+\",a[1].text)\n", "texto=re.findall(r\"\\w+\\s?\",texto[0])\n", "\" \".join(texto)" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "b'\\n\\n\\n\\n\\n\\n \\n \\n \\n Amazon.com.mx\\n \\n \\n \\n\\n\\n\\n\\n\\n\\n\\n
\\n\\n
\\n\\n
\\n\\n
\\n
\\n \\n

Ingresar los caracteres que aparecen a continuaci\\xc3\\xb3n

\\n

Sentimos las molestias. Necesitamos asegurarnos de que no eres un robot. Para obtener los mejores resultados, aseg\\xc3\\xbarate de que tu navegador acepta las cookies.

\\n
\\n
\\n\\n
\\n\\n
\\n
\\n\\n
\\n \\n
\\n
\\n
\\n

Ingresar los caracteres

\\n
\\n \\n
\\n
\\n
\\n
\\n
\\n
\\n Probar una imagen diferente\\n
\\n
\\n \\n
\\n
\\n
\\n
\\n\\n
\\n\\n
\\n \\n \\n \\n \\n \\n
\\n\\n
\\n
\\n\\n
\\n
\\n\\n
\\n\\n
\\n\\n
\\n\\n
\\n Condiciones de Uso\\n \\n \\n \\n \\n Aviso de Privacidad\\n
\\n
\\n \\xc2\\xa9 1996-2020, Amazon.com, Inc. o afiliados. Todos los derechos reservados.\\n \\n \\n
\\n
\\n\\n'" ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [ "link = \"https://www.amazon.com.mx/\"\n", "pagina = requests.get(link)\n", "#pagina.content\n", "pagina.content" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 4 }