使用.loc访问器的pandas日期时间索引的布尔掩码

•浏览 1

Boolean mask from pandas datetime index using .loc accessor

假设以下玩具代码：

import numpy as np

import pandas as pd

rng = pd.date_range('1/1/2011', periods=72, freq='H')

avec = np.random.rand(len(rng))

bvec = np.random.rand(len(rng))

df = pd.DataFrame({"A":avec,"B":bvec}, index=rng)
df.loc["2011-01-02",:]

array([False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False], dtype=bool)

(df.index >="2011-01-02") & (df.index <"2011-01-03")

import numpy as np

from datetime import datetime



(df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

(df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))
df = pd.concat([df]*1000)



%timeit (df.index >="2011-01-02") & (df.index <"2011-01-03")



%timeit (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))



assert ((df.index >="2011-01-02") & (df.index <"2011-01-03") == \\

    (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))).all()



# 1.21 ms ± 23 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

# 527 μs ± 11.3 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
df.index.isin(df.loc["2011-01-02",:].index)



array([False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False], dtype=bool)

# named slice of your original dataframe:

sliced_df = df.loc["2011-01-02",:]

# get boolean array:

df.index.isin(sliced_df.index)

我现在可以用

选择时间间隔的一部分

import numpy as np

import pandas as pd

rng = pd.date_range('1/1/2011', periods=72, freq='H')

avec = np.random.rand(len(rng))

bvec = np.random.rand(len(rng))

df = pd.DataFrame({"A":avec,"B":bvec}, index=rng)
df.loc["2011-01-02",:]

array([False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False], dtype=bool)

(df.index >="2011-01-02") & (df.index <"2011-01-03")

import numpy as np

from datetime import datetime



(df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

(df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))
df = pd.concat([df]*1000)



%timeit (df.index >="2011-01-02") & (df.index <"2011-01-03")



%timeit (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))



assert ((df.index >="2011-01-02") & (df.index <"2011-01-03") == \\

    (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))).all()



# 1.21 ms ± 23 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

# 527 μs ± 11.3 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
df.index.isin(df.loc["2011-01-02",:].index)



array([False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False], dtype=bool)

# named slice of your original dataframe:

sliced_df = df.loc["2011-01-02",:]

# get boolean array:

df.index.isin(sliced_df.index)

有没有办法有效地访问对应于结果切片的布尔掩码，即：

import numpy as np

import pandas as pd

rng = pd.date_range('1/1/2011', periods=72, freq='H')

avec = np.random.rand(len(rng))

bvec = np.random.rand(len(rng))

df = pd.DataFrame({"A":avec,"B":bvec}, index=rng)
df.loc["2011-01-02",:]

array([False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False], dtype=bool)

(df.index >="2011-01-02") & (df.index <"2011-01-03")

import numpy as np

from datetime import datetime



(df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

(df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))
df = pd.concat([df]*1000)



%timeit (df.index >="2011-01-02") & (df.index <"2011-01-03")



%timeit (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))



assert ((df.index >="2011-01-02") & (df.index <"2011-01-03") == \\

    (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))).all()



# 1.21 ms ± 23 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

# 527 μs ± 11.3 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
df.index.isin(df.loc["2011-01-02",:].index)



array([False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False], dtype=bool)

# named slice of your original dataframe:

sliced_df = df.loc["2011-01-02",:]

# get boolean array:

df.index.isin(sliced_df.index)

我已经尝试过早期 stackoverflow 答案中的建议，但 df.index.date 需要很长时间才能在我的数据集上运行...

如果性能很重要，链 2 布尔掩码：

import numpy as np

import pandas as pd

rng = pd.date_range('1/1/2011', periods=72, freq='H')

avec = np.random.rand(len(rng))

bvec = np.random.rand(len(rng))

df = pd.DataFrame({"A":avec,"B":bvec}, index=rng)
df.loc["2011-01-02",:]

array([False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False], dtype=bool)

(df.index >="2011-01-02") & (df.index <"2011-01-03")

import numpy as np

from datetime import datetime



(df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

(df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))
df = pd.concat([df]*1000)



%timeit (df.index >="2011-01-02") & (df.index <"2011-01-03")



%timeit (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))



assert ((df.index >="2011-01-02") & (df.index <"2011-01-03") == \\

    (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))).all()



# 1.21 ms ± 23 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

# 527 μs ± 11.3 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
df.index.isin(df.loc["2011-01-02",:].index)



array([False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False], dtype=bool)

# named slice of your original dataframe:

sliced_df = df.loc["2011-01-02",:]

# get boolean array:

df.index.isin(sliced_df.index)

您可以提取索引的 numpy 表示并与 np.datetime64 对象进行比较：

import numpy as np

import pandas as pd

rng = pd.date_range('1/1/2011', periods=72, freq='H')

avec = np.random.rand(len(rng))

bvec = np.random.rand(len(rng))

df = pd.DataFrame({"A":avec,"B":bvec}, index=rng)
df.loc["2011-01-02",:]

array([False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False], dtype=bool)

(df.index >="2011-01-02") & (df.index <"2011-01-03")

import numpy as np

from datetime import datetime



(df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

(df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))
df = pd.concat([df]*1000)



%timeit (df.index >="2011-01-02") & (df.index <"2011-01-03")



%timeit (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))



assert ((df.index >="2011-01-02") & (df.index <"2011-01-03") == \\

    (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))).all()



# 1.21 ms ± 23 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

# 527 μs ± 11.3 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
df.index.isin(df.loc["2011-01-02",:].index)



array([False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False], dtype=bool)

# named slice of your original dataframe:

sliced_df = df.loc["2011-01-02",:]

# get boolean array:

df.index.isin(sliced_df.index)

行为注意事项

上述解决方案特定于问题中的查询。正如@Jeff 指出的那样， datetime 的字符串表示使用部分索引。所以使用 numpy 应该只在特定情况下使用。

有关详细信息，请参阅关于 datetime 索引的 pandas 文档。

性能基准测试

import numpy as np

import pandas as pd

rng = pd.date_range('1/1/2011', periods=72, freq='H')

avec = np.random.rand(len(rng))

bvec = np.random.rand(len(rng))

df = pd.DataFrame({"A":avec,"B":bvec}, index=rng)
df.loc["2011-01-02",:]

array([False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False], dtype=bool)

(df.index >="2011-01-02") & (df.index <"2011-01-03")

import numpy as np

from datetime import datetime



(df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

(df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))
df = pd.concat([df]*1000)



%timeit (df.index >="2011-01-02") & (df.index <"2011-01-03")



%timeit (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))



assert ((df.index >="2011-01-02") & (df.index <"2011-01-03") == \\

    (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))).all()



# 1.21 ms ± 23 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

# 527 μs ± 11.3 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
df.index.isin(df.loc["2011-01-02",:].index)



array([False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False], dtype=bool)

# named slice of your original dataframe:

sliced_df = df.loc["2011-01-02",:]

# get boolean array:

df.index.isin(sliced_df.index)

IIUC，你可以这样做：

import numpy as np

import pandas as pd

rng = pd.date_range('1/1/2011', periods=72, freq='H')

avec = np.random.rand(len(rng))

bvec = np.random.rand(len(rng))

df = pd.DataFrame({"A":avec,"B":bvec}, index=rng)
df.loc["2011-01-02",:]

array([False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False], dtype=bool)

(df.index >="2011-01-02") & (df.index <"2011-01-03")

import numpy as np

from datetime import datetime



(df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

(df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))
df = pd.concat([df]*1000)



%timeit (df.index >="2011-01-02") & (df.index <"2011-01-03")



%timeit (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))



assert ((df.index >="2011-01-02") & (df.index <"2011-01-03") == \\

    (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))).all()



# 1.21 ms ± 23 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

# 527 μs ± 11.3 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
df.index.isin(df.loc["2011-01-02",:].index)



array([False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False], dtype=bool)

# named slice of your original dataframe:

sliced_df = df.loc["2011-01-02",:]

# get boolean array:

df.index.isin(sliced_df.index)

但是，@jezrael 的回答更快。我将此作为替代解决方案。这提供的唯一便利是如果您想按名称而不是日期字符串引用切片数据帧，例如：

import numpy as np

import pandas as pd

rng = pd.date_range('1/1/2011', periods=72, freq='H')

avec = np.random.rand(len(rng))

bvec = np.random.rand(len(rng))

df = pd.DataFrame({"A":avec,"B":bvec}, index=rng)
df.loc["2011-01-02",:]

array([False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, True, True, True, True, True, True,

    True, True, True, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False,

   False, False, False, False, False, False, False, False, False], dtype=bool)

(df.index >="2011-01-02") & (df.index <"2011-01-03")

import numpy as np

from datetime import datetime



(df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

(df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))
df = pd.concat([df]*1000)



%timeit (df.index >="2011-01-02") & (df.index <"2011-01-03")



%timeit (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))



assert ((df.index >="2011-01-02") & (df.index <"2011-01-03") == \\

    (df.index.values >= np.datetime64(datetime.strptime("2011-01-02", '%Y-%m-%d'))) & \\

    (df.index.values < np.datetime64(datetime.strptime("2011-01-03", '%Y-%m-%d')))).all()



# 1.21 ms ± 23 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

# 527 μs ± 11.3 μs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
df.index.isin(df.loc["2011-01-02",:].index)



array([False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, True, True, True, True, True, True,

  True, True, True, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False,

 False, False, False, False, False, False, False, False, False], dtype=bool)

# named slice of your original dataframe:

sliced_df = df.loc["2011-01-02",:]

# get boolean array:

df.index.isin(sliced_df.index)
赞 (0)
« 上一篇
下一篇 »

相关推荐

  • Spring部署设置openshift

    Springdeploymentsettingsopenshift我有一个问题让我抓狂了三天。我根据OpenShift帐户上的教程部署了spring-eap6-quickstart代码。我已配置调试选项，并且已将Eclipse工作区与OpehShift服务器同步-服务器上的一切工作正常，但在Eclipse中出现无法消除的错误。我有这个错误：cvc-complex-type.2.4.a:Invali…
    2025-04-161

  • 检查Java中正则表达式中模式的第n次出现

    CheckfornthoccurrenceofpatterninregularexpressioninJava本问题已经有最佳答案，请猛点这里访问。我想使用Java正则表达式检查输入字符串中特定模式的第n次出现。你能建议怎么做吗？这应该可以工作：MatchResultfindNthOccurance(intn,Patternp,CharSequencesrc){Matcherm=p.matcher…
    2025-04-161

  • 如何让 JTable 停留在已编辑的单元格上

    HowtohaveJTablestayingontheeditedcell如果有人编辑JTable的单元格内容并按Enter，则内容会被修改并且表格选择会移动到下一行。是否可以禁止JTable在单元格编辑后转到下一行？原因是我的程序使用ListSelectionListener在单元格选择上同步了其他一些小部件，并且我不想在编辑当前单元格后选择下一行。Enter的默认绑定是名为selectNext…
    2025-04-161

  • Weblogic 12c 部署

    Weblogic12cdeploy我正在尝试将我的应用程序从Tomcat迁移到Weblogic12.2.1.3.0。我能够毫无错误地部署应用程序，但我遇到了与持久性提供程序相关的运行时错误。这是堆栈跟踪：javax.validation.ValidationException:CalltoTraversableResolver.isReachable()threwanexceptionatorg.…
    2025-04-161

  • Resteasy Content-Type 默认值

    ResteasyContent-Typedefaults我正在使用Resteasy编写一个可以返回JSON和XML的应用程序，但可以选择默认为XML。这是我的方法：@GET@Path("/content")@Produces({MediaType.APPLICATION_XML,MediaType.APPLICATION_JSON})publicStringcontentListRequestXm…
    2025-04-161

  • 代码不会停止运行，在 Java 中

    thecodedoesn'tstoprunning,inJava我正在用Java解决项目Euler中的问题10，即"Thesumoftheprimesbelow10is2+3+5+7=17.Findthesumofalltheprimesbelowtwomillion."我的代码是packageprojecteuler_1;importjava.math.BigInteger;importjava…
    2025-04-161

  • Out of memory java heap space

    Outofmemoryjavaheapspace我正在尝试将大量文件从服务器发送到多个客户端。当我尝试发送大小为700mb的文件时，它显示了"OutOfMemoryjavaheapspace"错误。我正在使用Netbeans7.1.2版本。我还在属性中尝试了VMoption。但仍然发生同样的错误。我认为阅读整个文件存在一些问题。下面的代码最多可用于300mb。请给我一些建议。提前致谢publicc…
    2025-04-161

  • Log4j 记录到共享日志文件

    Log4jLoggingtoaSharedLogFile有没有办法将log4j日志记录事件写入也被其他应用程序写入的日志文件。其他应用程序可以是非Java应用程序。有什么缺点？锁定问题？格式化？Log4j有一个SocketAppender，它将向服务发送事件，您可以自己实现或使用与Log4j捆绑的简单实现。它还支持syslogd和Windows事件日志，这对于尝试将日志输出与来自非Java应用程序…
    2025-04-161
手机版技术脚本