82 seconds to extract one row in the db, java.lang.OutOfMemoryError: GC overhead limit exceeded, with large database
82 seconds to extract one row in the db, java.lang.OutOfMemoryError: GC overhead limit exceeded, with large database
我在数据库(本地主机)的 table 中添加了 100 000 行,从那时起我得到 this error:
java.lang.OutOfMemoryError: GC overhead limit exceeded
我通过输入 consol 解决了这个问题:
javaw -XX:-UseConcMarkSweepGC
控制台的输出是(有关上下文,请参阅下面的代码):
2015-08-02T02:57:22.779+0200|Info: 5
2015-08-02T02:57:22.779+0200|Info: end, time taken: 82755
在数据库中提取一行需要82秒(见最后的代码)。
当我的行数较少时它工作正常,所以我想知道:
- 为什么提取 1 行要花这么多时间? JPA 不可能提取对象中的每一行?或者是吗?哇哦。
- 有办法解决这个问题吗?我的意思是在 80 秒内提取一行是非常慢的。
- 我真的必须输入命令
-XX:-UseConcMarkSweepGC
吗?
它有什么作用 ?来自 the doc :
Use concurrent mark-sweep collection for the old generation.
(Introduced in 1.4.1)
我的代码是这样的:
@EJB
private ThreadLookUpInterface ts;
@Schedule(hour = "*", minute = "*/1", second = "0", persistent = false)
@Override
public void makeTopThreadList() {
System.out.println("" + ts.getThread(5).getIdthread());
}
我的服务 ejb 是这样的:
@Stateless
public class ThreadLookUpService implements ThreadLookUpInterface {
@PersistenceContext(unitName = "my-pu")
private EntityManager em;
private static final String FIND_THREAD_BY_ID = "SELECT t FROM Thethread t WHERE t.idthread=:id";
@Override
public Thethread getThread(int threadId) {
Query query = em.createQuery(FIND_THREAD_BY_ID);
query.setParameter("id", threadId);
try {
Thethread thread = (Thethread) query.getSingleResult();
return thread;
} catch (NoResultException e) {
return null;
} catch (Exception e) {
throw new DAOException(e);
}
}
}
我的实体:
@Entity
@Table(name = "thethreads")
@NamedQuery(name = "Thethread.findAll", query = "SELECT t FROM Thethread t")
public class Thethread implements Serializable {
private static final long serialVersionUID = 1L;
@Id
private int idthread;
private String content;
@Temporal(TemporalType.TIMESTAMP)
@Column(name = "date_posted")
private Date datePosted;
private int downvotes;
@Column(name = "hot_score")
private int hotScore;
@Column(name = "is_pol")
private String isPol;
private String title;
private String type;
private int upvotes;
// bi-directional many-to-one association to Category
@OneToMany(mappedBy = "thethread", fetch = FetchType.EAGER)
private List<Category> categories;
// bi-directional many-to-one association to Post
@OneToMany(mappedBy = "thethread", fetch = FetchType.EAGER)
private List<Post> posts;
// bi-directional many-to-one association to Category
@ManyToOne
@JoinColumn(name = "categories_idcategory")
private Category category;
// bi-directional many-to-one association to Post
@ManyToOne
@JoinColumn(name = "last_post")
private Post post;
// bi-directional many-to-one association to User
@ManyToOne
@JoinColumn(name = "posted_by")
private User user;
// bi-directional many-to-one association to ThreadVote
@OneToMany(mappedBy = "thethread", fetch = FetchType.EAGER)
private List<ThreadVote> threadVotes;
public Thethread() {
}
}
command -XX:-UseConcMarkSweepGC? What does it do?
设置垃圾收集器:ConcurrentMarkSweep (CMS)。
Java 堆(对象在其生命周期中所在的位置)主要分为两部分:年轻代和老年代。垃圾收集器负责在该分区上应用一些算法或策略来清理堆。
CMS 不像默认 GC 那样在完全垃圾收集期间停止应用程序线程,而是使用一个或多个后台线程定期扫描 旧代 并丢弃未使用的对象。
这可以帮助您减少开销情况,但问题并没有消失。
extract every row in an object ? Or does it ? Just wow.Is there a way around this ? I mean extracting a single row in 80 seconds is borderline slow.
答案首先取决于您的实体模型。例如,一个常见的问题是滥用 Eager fetch 类型,这会导致 fetch/retrieve 大量不必要的对象为此执行多个查询语句。
还取决于您的 JPA 实现和数据库如何解决任务,但我建议您开始查看实体模型。也许如果你 post Thethread
实体有人可以识别可能的问题。
我在数据库(本地主机)的 table 中添加了 100 000 行,从那时起我得到 this error:
java.lang.OutOfMemoryError: GC overhead limit exceeded
我通过输入 consol 解决了这个问题:
javaw -XX:-UseConcMarkSweepGC
控制台的输出是(有关上下文,请参阅下面的代码):
2015-08-02T02:57:22.779+0200|Info: 5
2015-08-02T02:57:22.779+0200|Info: end, time taken: 82755
在数据库中提取一行需要82秒(见最后的代码)。 当我的行数较少时它工作正常,所以我想知道:
- 为什么提取 1 行要花这么多时间? JPA 不可能提取对象中的每一行?或者是吗?哇哦。
- 有办法解决这个问题吗?我的意思是在 80 秒内提取一行是非常慢的。
- 我真的必须输入命令
-XX:-UseConcMarkSweepGC
吗? 它有什么作用 ?来自 the doc :
Use concurrent mark-sweep collection for the old generation. (Introduced in 1.4.1)
我的代码是这样的:
@EJB
private ThreadLookUpInterface ts;
@Schedule(hour = "*", minute = "*/1", second = "0", persistent = false)
@Override
public void makeTopThreadList() {
System.out.println("" + ts.getThread(5).getIdthread());
}
我的服务 ejb 是这样的:
@Stateless
public class ThreadLookUpService implements ThreadLookUpInterface {
@PersistenceContext(unitName = "my-pu")
private EntityManager em;
private static final String FIND_THREAD_BY_ID = "SELECT t FROM Thethread t WHERE t.idthread=:id";
@Override
public Thethread getThread(int threadId) {
Query query = em.createQuery(FIND_THREAD_BY_ID);
query.setParameter("id", threadId);
try {
Thethread thread = (Thethread) query.getSingleResult();
return thread;
} catch (NoResultException e) {
return null;
} catch (Exception e) {
throw new DAOException(e);
}
}
}
我的实体:
@Entity
@Table(name = "thethreads")
@NamedQuery(name = "Thethread.findAll", query = "SELECT t FROM Thethread t")
public class Thethread implements Serializable {
private static final long serialVersionUID = 1L;
@Id
private int idthread;
private String content;
@Temporal(TemporalType.TIMESTAMP)
@Column(name = "date_posted")
private Date datePosted;
private int downvotes;
@Column(name = "hot_score")
private int hotScore;
@Column(name = "is_pol")
private String isPol;
private String title;
private String type;
private int upvotes;
// bi-directional many-to-one association to Category
@OneToMany(mappedBy = "thethread", fetch = FetchType.EAGER)
private List<Category> categories;
// bi-directional many-to-one association to Post
@OneToMany(mappedBy = "thethread", fetch = FetchType.EAGER)
private List<Post> posts;
// bi-directional many-to-one association to Category
@ManyToOne
@JoinColumn(name = "categories_idcategory")
private Category category;
// bi-directional many-to-one association to Post
@ManyToOne
@JoinColumn(name = "last_post")
private Post post;
// bi-directional many-to-one association to User
@ManyToOne
@JoinColumn(name = "posted_by")
private User user;
// bi-directional many-to-one association to ThreadVote
@OneToMany(mappedBy = "thethread", fetch = FetchType.EAGER)
private List<ThreadVote> threadVotes;
public Thethread() {
}
}
command -XX:-UseConcMarkSweepGC? What does it do?
设置垃圾收集器:ConcurrentMarkSweep (CMS)。 Java 堆(对象在其生命周期中所在的位置)主要分为两部分:年轻代和老年代。垃圾收集器负责在该分区上应用一些算法或策略来清理堆。 CMS 不像默认 GC 那样在完全垃圾收集期间停止应用程序线程,而是使用一个或多个后台线程定期扫描 旧代 并丢弃未使用的对象。 这可以帮助您减少开销情况,但问题并没有消失。
extract every row in an object ? Or does it ? Just wow.Is there a way around this ? I mean extracting a single row in 80 seconds is borderline slow.
答案首先取决于您的实体模型。例如,一个常见的问题是滥用 Eager fetch 类型,这会导致 fetch/retrieve 大量不必要的对象为此执行多个查询语句。
还取决于您的 JPA 实现和数据库如何解决任务,但我建议您开始查看实体模型。也许如果你 post Thethread
实体有人可以识别可能的问题。