Python - 数据框 url 解析问题

Question

我正在尝试将 url 中的域名从一列获取到另一列。它在类似对象的字符串上工作，当我应用于数据框时它不起作用。如何将其应用于数据框？

尝试过：

from urllib.parse import urlparse
import pandas as pd
id1 = [1,2,3]
ls = ['https://google.com/tensoflow','https://math.com/some/website',np.NaN]
df = pd.DataFrame({'id':id1,'url':ls})
df
# urlparse(df['url']) # ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
# df['url'].map(urlparse) # AttributeError: 'float' object has no attribute 'decode'

处理字符串：

string = 'https://google.com/tensoflow'
parsed_uri = urlparse(string)
result = '{uri.scheme}://{uri.netloc}/'.format(uri=parsed_uri)
result

寻找专栏：

col3
https://google.com/
https://math.com/
nan

错误

Answer 1

你可以试试这个。

Here I have used pandas.Series.apply() to solve.

» 初始化和导入

>>> from urllib.parse import urlparse
>>> import pandas as pd
>>> id1 = [1,2,3]
>>> import numpy as np
>>> ls = ['https://google.com/tensoflow','https://math.com/some/website',np.NaN]
>>> ls
['https://google.com/tensoflow', 'https://math.com/some/website', nan]
>>>

» 检查新创建的 DataFrame。

>>> df = pd.DataFrame({'id':id1,'url':ls})
>>> df
   id                            url
0   1   https://google.com/tensoflow
1   2  https://math.com/some/website
2   3                            NaN
>>> 
>>> df["url"]
0     https://google.com/tensoflow
1    https://math.com/some/website
2                              NaN
Name: url, dtype: object
>>>

» 在 url 列上使用 pandas.Series.apply(func) 应用函数..

>>> df["url"].apply(lambda url: "{uri.scheme}://{uri.netloc}/".format(uri=urlparse(url)) if not pd.isna(url) else np.nan)
0    https://google.com/
1      https://math.com/
2                    NaN
Name: url, dtype: object
>>> 
>>> df["url"].apply(lambda url: "{uri.scheme}://{uri.netloc}/".format(uri=urlparse(url)) if not pd.isna(url) else str(np.nan))
0    https://google.com/
1      https://math.com/
2                    nan
Name: url, dtype: object
>>> 
>>>

» 把上面的结果存到一个变量中（不是必须的，只是为了简单）。

>>> s = df["url"].apply(lambda url: "{uri.scheme}://{uri.netloc}/".format(uri=urlparse(url)) if not pd.isna(url) else str(np.nan))
>>> s
0    https://google.com/
1      https://math.com/
2                    nan
Name: url, dtype: object
>>>

» 最后

>>> df2 = pd.DataFrame({"col3": s})
>>> df2
                  col3
0  https://google.com/
1    https://math.com/
2                  nan
>>>

» 为确定什么是 s 和什么是 df2，请检查类型（同样，不是强制性的）。

>>> type(s)
<class 'pandas.core.series.Series'>
>>> 
>>> 
>>> type(df2)
<class 'pandas.core.frame.DataFrame'>
>>>

参考链接：

Python - 数据框 url 解析问题

Python - dataframe url parsing issue

urllib

python-3.x