中吴南顾惟一笑

成功法则就是那19个字

日志

关于我

guyot

文章分类

流数据挖掘与DSMS

2010-06-12 12:46:33| 分类： dbms | 标签： |举报 |字号大中小订阅

下载LOFTER 我的照片书 |

特点
大数据量，甚至无限
频繁的变化和快速的响应
线性扫描算法，查询次数有限
random access is expensive

DBMS
持久的关系
One-time queries
随机的访问
“无限”的磁盘空间
当前状态有效
相对较低的更新率
很少“实时服务”
假定数据精确无误
访问策略由查询处理器在数据库设计时确定

DSMS(Data Stream Management System)
瞬间的流
连续的查询
序列化的访问
有限的主存
数据的到达顺序是关键
数据传输率未知
实时响应
过时/模糊的数据
变化的数据及数据量

目前的DSMS项目
STREAM (Stanford): A general-purpose DSMS
Cougar (Cornell): sensors
Aurora (Brown/MIT): sensor monitoring, dataflow
Hancock (AT&T): telecom streams
Niagara (OGI/Wisconsin): Internet XML databases
OpenCQ (Georgia Tech): triggers, incr. view maintenance
Tapestry (Xerox): pub/sub content-based filtering
Telegraph (Berkeley): adaptive engine for sensors
Tradebot (www.tradebot.com): stock tickers & streams
Tribeca (Bellcore): network monitoring
Streaminer (UIUC): new project for stream data mining

应用实例
网络安全
   数据包流，用户的会话信息
   查询: URL 过滤，异常监测，网络攻击和病毒来源
金融领域
   交易数据流，股票行情，消息反馈
   查询: 套汇可能性分析，模式

研究方向
流数据建模(Stream data model)
   STanford stREam datA Manager (STREAM)
   Data Stream Management System (DSMS)
流检索/查询建模(Stream query model)
   Continuous Queries
   Sliding windows
流数据挖掘(Stream data mining)
   Clustering & summarization (Guha, Motwani et al.)
   Correlation of data streams (Gehrke et al.)
   Classification of stream data (Domingos et al.)

评论这张

转发至微博

阅读(633)| 评论(0)

历史上的今天

this.p={  m:2,
              b:2,
              loftPermalink:'',
              id:'fks_080074085081081074080087083095085084081070083082086065085',
              blogTitle:'流数据挖掘与DSMS',
              blogAbstract:'特点<br\>大数据量，甚至无限<br\>频繁的变化和快速的响应<br\>线性扫描算法，查询次数有限<br\>random access is expensive<br\><br\>DBMS<br\>持久的关系<br\>One-time queries<br\>随机的访问<br\>“无限”的磁盘空间<br\>当前状态有效<br\>相对较低的更新率<br\>很少“实时服务”<br\>假定数据精确无误<br\>访问策略由查询处理器在数据库设计时确定<br\><br\><br\>DSMS(Data Stream Management System)<br\>瞬间的流 <br\>连续的查询<br\>序列化的访问<br\>有限的主存<br\>数据的到达顺序是关键<br\>数据传输率未知<br\>实时响应<br\>过时/模糊的数据<br\>变化的数据及数据量<br\><br\><br\>目前的DSMS项目<br\>STREAM (Stanford): A general-purpose DSMS',
              blogTag:'',
              blogUrl:'blog/static/120574021201051204633405',
              isPublished:1,
              istop:false,
              type:0,
              modifyTime:0,
              publishTime:1276317993405,
              permalink:'blog/static/120574021201051204633405',
              commentCount:0,
              mainCommentCount:0,
              recommendCount:0,
              bsrk:-100,
              publisherId:0,
              recomBlogHome:false,
              currentRecomBlog:false,
              attachmentsFileIds:[],
              vote:{},
              groupInfo:{},
              friendstatus:'none',
              followstatus:'unFollow',
              pubSucc:'',
              visitorProvince:'',
              visitorCity:'',
              visitorNewUser:false,
              postAddInfo:{},
              mset:'000',
              mcon:'',
              srk:-100,
              remindgoodnightblog:false,
              isBlackVisitor:false,
              isShowYodaoAd:false,
              hostIntro:'',
              hmcon:'0',
              selfRecomBlogCount:'0',
              lofter_single:'<iframe width="140" height="560" style="overflow:hidden;" src="http://www.lofter.com/mailEntry.do?blogad=1&blog" frameBorder="0"></iframe>'
            }

{list a as x}
    {if !!x}
    <div class="iblock nbw-fce nbw-f40">
      <a class="fc03 noul" target="_blank" hidefocus="true" href="http://blog.163.com/${x.visitorName}/">
      {if x.visitorName==visitor.userName}
      <img alt="${x.visitorNickname|escape}" onerror="this.src=location.f40" class="cwd bdwa bdc0" src="${fn1(x.visitorName)}&r=${visitor.imageUpdateTime}"/>
      {else}
      <img alt="${x.visitorNickname|escape}" onerror="this.src=location.f40" class="cwd bdwa bdc0" src="${fn1(x.visitorName)}"/>
      {/if}
      </a>
      <div class="cwd vname thide">
        {if x.moveFrom=='wap'}
          <a class="noul pnt" target="_blank" href="http://blog.163.com/services/wapblog.html?frompersonalbloghome"><span title="来自网易手机博客" class="iblock wapIcon"> </span></a>
        {elseif x.moveFrom=='iphone'}
          <a class="noul pnt" target="_blank"><span title="来自iPhone客户端" class="iblock iphoneIcon"> </span></a>
        {elseif x.moveFrom=='android'}
          <a class="noul pnt" target="_blank"><span title="来自Android客户端" class="iblock androidIcon"> </span></a>
        {elseif x.moveFrom=='mobile'}
          <a class="noul pnt" target="_blank" href="http://blog.163.com/services/emsblog.html?frompersonalbloghome"><span title="来自网易短信写博" class="iblock wapIcon"> </span></a>
        {/if}
        <a class="fc03 m2a"  target="_blank" hidefocus="true" href="http://blog.163.com/${x.visitorName}/">
          ${fn(x.visitorNickname,8)|escape}
        </a>
      </div>
    </div>
    {/if}
    {/list}

<#--最新日志，群博日志--> <#--推荐日志-->

<p class="fc06">推荐过这篇日志的人：</p>
    <div>
      {list a as x}
      {if !!x}
      <div class="iblock nbw-fce nbw-f40">
        <a class="fc03 noul" target="_blank" hidefocus="true" href="http://blog.163.com/${x.recommenderName}/">
        <img alt="${x.recommenderNickname|escape}" onerror="this.src=location.f40" class="cwd bdwa bdc0" src="${fn1(x.recommenderName)}"/>
        </a>
        <div class="cwd thide">
          <a class="fc03 m2a" target="_blank" hidefocus="true" href="http://blog.163.com/${x.recommenderName}/">
            ${fn(x.recommenderNickname,6)|escape}
          </a>
        </div>
      </div>
      {/if}
      {/list}
    </div>
    {if !!b&&b.length>0}
    <p  class="fc06">他们还推荐了：</p>
    <ul>
    {list b as y}
      {if !!y}
        <li class="rrb"><span class="iblock">·</span><a class="fc03 m2a" target="_blank" href="http://blog.163.com/${y.recommendBlogPermalink}/?from=blog/static/120574021201051204633405">${y.recommendBlogTitle|escape}</a></li>
      {/if}
    {/list}
    </ul>
    {/if}

<#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇，下一篇--> <#-- 热度 -->

{list a as x}
    {if !!x}
    <div class="hotItem iblock nbw-fce nbw-f40">
      <a class="fc03 noul" target="_blank" hidefocus="true" href="http://blog.163.com/${x.publisherUsername}/">
      {if x.publisherUsername==visitor.userName}
      <img alt="${x.publisherNickname|escape}" onerror="this.src=location.f40" class="cwd bdwa bdc0" src="${fn1(x.publisherUsername)}&r=${visitor.imageUpdateTime}"/>
      {else}
      <img alt="${x.publisherNickname|escape}" onerror="this.src=location.f40" class="cwd bdwa bdc0" src="${fn1(x.publisherUsername)}"/>
      {/if}
      </a>
      <div class="cwd vname thide">
        <a class="fc03 m2a"  target="_blank" hidefocus="true" href="http://blog.163.com/${x.publisherUsername}/">
          ${fn(x.publisherNickname,8)|escape}
        </a>
      </div>
      <a class="f-myLikeIcons hottype {if x.type==1} js-liketype{elseif x.type==2} js-reblogtype{elseif x.type==3} js-sharetype{else}{/if}" target="_blank" hidefocus="true" href="http://blog.163.com/${x.publisherUsername}/"> </a>
    </div>
    {/if}
    {/list}

<#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->

页脚

我的照片书 - 手机博客 - 下载LOFTER APP - 订阅此博客

中吴南顾惟一笑

导航

日志

流数据挖掘与DSMS

历史上的今天

最近读者

热度

评论

页脚